Data center cooling

ABSTRACT

A system includes a substantially sealed, substantially airtight cabinet sized for housing a vertical array of heat-producing units, the cabinet having an exterior shell and the system including an interior divider wall disposed inside the cabinet, the shell and divider wall providing an equipment chamber adapted to support the array such that the array cooperates with the shell and divider wall in use to define a first plenum, the first plenum having a first inlet defined by the divider wall for receiving a flow of cooling gas and having a first outlet defined by a plurality of openings through the array whereby the first plenum communicates with the openings in use to exhaust substantially all of the flow of cooling fluid through the openings and hence through the array, wherein the divider wall is configured to allow the first inlet to admit the gas to the first plenum in a substantially horizontal direction.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 12/406,015 filed Mar. 17, 2009, which is acontinuation application of U.S. patent application Ser. No. 10/509,434filed Jun. 6, 2005, which claims priority under 35 USC §371 toApplication No. PCT/GB03/01367, filed Mar. 28, 2003, and to UnitedKingdom Application No. 02073823, filed Mar. 28, 2002, all of which areincorporated by reference in their entireties for all purposes.

FIELD OF THE INVENTION

The invention relates to cooling techniques and in particular to coolingdata centers.

BACKGROUND

This disclosure relates to the art of data centers, and in particular todata processing centers in which banks of servers or other equipment arehoused in a protected environment. Specific aspects of the disclosurerelate to housings or cabinets for electronic equipment for use in bothcontrolled environments (e.g. computer/data rooms) and non-controlledenvironments (e.g. ordinary offices, factories, external sites etc.).Although currently intended for housing electronic equipment, thedisclosure is not limited to this use and instead can be utilized withrespect to any equipment for which forced air cooling is useful.

Data centers are important business facilities which aim to provideprotected environments for housing electronic equipment, such ascomputer and telecommunications systems, for a wide range ofapplications. With ever-increasing numbers of individuals and businessesrelying on the internet, hence giving rise to e-facilities such asapplication service providers, internet service providers, networkoperation centers, and co-location and web-hosting sites, data centersare becoming busier and more common. In particular, the growth of theinternet has resulted in unprecedented levels of server-based computing.Providers have found that many of their network infrastructure and webapplications work best on dedicated servers.

While embodiments of the disclosure will be described in the context ofinternet-related equipment, broader applicability is within the scope ofthe disclosure. References to data centers in this specification aretherefore to be construed broadly to encompass installations that do notnecessarily relate to the internet, such as those involvingtelecommunications, or any other equipment assembly using forced aircooling.

Computer and telecommunications systems are commonly gathered into datacenters because their sensitive electronics require protection fromhazards in the surrounding environment such as air-borne dust, spillagesand fluctuations in temperature and humidity, as well as from theever-present danger of power disturbances such as outages, surges andspikes. Flooding and fire monitoring/suppression systems are alsorequired. It makes commercial sense that these protective facilities areshared between users or tenants of data centers, although some datacenters are devoted to a single corporate user.

The equipment housed within data centers can be very valuable, but thedata stored by that equipment is potentially of much greater value, asis the cost of downtime should the equipment fail. Consequently, theoperators of data centers assume a great responsibility for ensuring theprotection and continuous fault-free running of the equipment that theyhouse. Tenants can be expected to claim substantial damages if thoseobjectives are not met.

In recent years, items of electronic equipment such as servers haveshrunk in size to the extent of being suitable for rack mounting. Now,therefore, servers in a data center are usually housed in equipmentracks or cabinets of a generally standard size and shape enablingservers and their supporting equipment to be housed in a modular,interchangeable fashion. Racks or cabinets are typically supported onraised floors beneath which complex cable networks for electricalinterconnection for both power supply and system communication can belaid while allowing access for maintenance and re-routing.

An equipment rack is an open frame with a system of uprights havingholes spaced at set modular centers, which are referred to as unitspacings or, for short, U's, 1U represents a vertical spacing of 1.75inches (44.45 mm). The width between the uprights (a unit spacing width)usually conforms to a standard of either 19 inches (483 mm) or 27 inches(675 mm). Electronic equipment is typically manufactured in a chassisform for rack mounting in accordance with these standard modules,although non-standard sizes are of course possible for specificapplications.

An equipment cabinet is essentially a rack as described above butmounted inside an enclosure. The cabinet has access doors at front andrear to allow maintenance access to the equipment within and provides adegree of physical security to the equipment. A typical so-called‘standard cabinet’ would have an external width of either 600 mm or 800mm, an external depth of either 800 mm or 900 mm, and an external heightof at least 2100 mm. Such a cabinet would be capable of accommodating astack of 42 to 45 1U units and so would be termed a 42U or 45U cabinetas appropriate.

Units mounted within a rack or cabinet need not necessarily be serverunits: for example, uninterruptible power supply (UPS) units are ofteninstalled to maintain and smooth power supply to other units.

The access doors of a cabinet may be solid, glazed, perforated or acombination of these, and will usually be lockable by means of keys ordigital keypads. More sophisticated locks relying upon scanners such asthumbprint or palm readers are also possible.

Equipment racks or cabinets are typically arranged in rows within aso-called technical space in a data center with an aisle space betweenthem of approximately 1200 mm and sometimes as small as 900 mm. Thisaisle space, also known in the art as ‘white space’, affords access totechnical personnel for the purposes of maintenance, monitoring,installation and so on.

As mentioned above, some data centers may be shared by several tenantswho house equipment there and so require access to the center. Whilethis raises security concerns, these concerns may be partially overcomeby restricted personnel access to the technical space. However, the moreaccess takes place, the more difficult it becomes to maintain a closedenvironment in which temperature, humidity and ingress of dust or othercontaminants can be controlled. For example, to achieve a cleanenvironment IP (ingress protection) rating under British and Europeanstandards, a sealed filtered system is required which is difficult whenthe sealed system is an entire room. Also, a multi-tenant facility alsoincreases the chances of accidental damage to equipment, such as impactdamage or spillage of liquids.

Smoke and fire detectors and fire fighting capabilities are importantfeatures of data centers. Early fire detection and efficient firesuppressant systems are vital for minimizing equipment damage, dataloss, system downtime and service interruption in the event of an actualor impending fire.

To minimize equipment damage, many users and operators prefer inertgases to water sprinkler systems for dousing fires. However, in recentyears, some operators have omitted gas protection due to the highcapital cost of the system and the high cost of recharging the systemonce discharged. Many gases are also environmentally unsound. In knownenvironmentally conditioned data center systems, the gas or water systemnormally discharges into the technical space as a whole and the entireroom is closed off, and the equipment within shut down, while theparticular server or other equipment unit at fault is detected andremoved.

Equipment manufacturers and industry standards specify tight tolerancesfor environmental conditions to ensure optimal performance of theequipment. For example, relatively small but sudden fluctuations fromthe recommended operating temperature (e.g. at a rate of temperaturechange of as little as 10° C. per hour) can cause thermal shock anddamage delicate circuits. High humidity can corrode switching circuitrycausing malfunctions and equipment failures, whereas low humidity canpromote static electricity that interferes with proper equipmentoperation.

The environmental conditions of a data center are largely determined bya combination of the equipment heat load in the room and the temperatureand humidity loads resulting from infiltration of outside air. Otherload factors include people working within the technical space, whointroduce heat and humidity, and lighting of the technical space, whichintroduces radiant and convective heat. However, the dominant challengein environmental control of the technical space is the generation ofheat by the electronic equipment housed within.

The heat generated by electronic equipment is related to the powerconsumed by that equipment. New designs of electronic equipment whichare more compact than previous models tend to have higher powerconsumption and therefore a greater heat output. In particular, thedesire for compactness has been driven by the commercial need to fit asmany servers as possible into existing data centers. Smaller servers anda denser population of servers are required to return as much revenue aspossible per square meter of rackable area within the data center.

To this end, servers have been designed that fit into 1U of space; these‘1U servers’ are also referred to as ‘high density’ servers. Suchservers are relatively heavy and have a high heat output of up to 1000BTU's per hour (293 W) per server, the level of which depends on theserver configuration in terms of number of processors, hard drives etc.,and the software type and data being processed. So, while racks orcabinets can in theory be filled with such high density equipment, inreality, overheating may occur in that event. Servers present aconsiderably greater challenge in this respect than other equipment aptto be rack-mounted, such as UPS units.

Overheating has become a major issue since high density serverdeployment began. As recently as three years ago, typical electricalloads were between 300-400 W/m2 of rackable space within the technicalarea but today 1200 W/m2 is the average with some installations being ashigh as 2000 W/m2. This increase in power consumption is reflected inheat output within the technical space, which in turn adds to thecooling load and hence to the overall power requirement of the datacenter. Indeed, the overall power requirement has almost doubled overthe three-year period. This presents another challenge in terms ofinadequate power supply.

Those designing cooling systems for data environments are faced with notjust the problem of ever greater cooling load requirements, but withpredicting the size of the load to be allowed for at any given point oftime. Moore's Law predicts the doubling of semiconductor performanceevery eighteen months. If Moore's Law continues to hold (and it isanticipated that it will, at least through 2005), there will continue tobe a dramatic and continuing increase in product power densities,coupled with the design of smaller devices which is now being referredto as the problem of ‘compaction’.

With compaction comes an increasing amount of cabling to make theconnections to the greater number of smaller devices housed in a rack.New category cables such as ‘Cat 6’ are larger in diameter than those ofearlier generations, which both restricts airflow within the rack andadds to the total rack weight. The electronic equipment itself now tendsto be provided with dual-corded power supplies, or triple-corded in somecases. The power supplies, often ‘hot pluggable’, add to the deviceweight and therefore to the overall rack weight. In fact, some rackshave to be braced as they near their maximum slenderness ratio.

The risk of overheating means that cabinets are very often leftunfilled: they are not fully populated with servers or other equipment,meaning that some available levels remain unused. This is to thedetriment of efficient space utilization within the data center and,ultimately, increases the cost of housing each server because fewerservers or fewer tenants share the infrastructure costs of the datacenter.

In existing data centers, temperature regulation is commonly achieved byclose control room air conditioning units (also known as computer roomair conditioning or CRAC units) within the technical space. An exampleof such a conventionally cooled data center is shown in FIG. 1.

The conventional computer/data room cooling technique illustrated inFIG. 1 is the typical approach to cooling electronic equipment currentlyemployed. Within an enclosed room 1 defined by a room floor 5, sidewalls 3 and a ceiling 4, a suspended raised floor 2 is installed at apre-determined height above the base floor 5. The suspended floor 2 andthe base floor 5 together constitute a double floor structure defining afree space or floor void 6 which is used as an air passageway and oftenas a technical service zone for power and data cables. The raised floorstructure 2 comprises a plurality of panels which permit access to thefloor void 6 from above.

Sometimes a suspended ceiling 7 having a plurality of ceiling panels isprovided below the base ceiling 4. The suspended ceiling 7 and the baseceiling 4 combine to form a double ceiling structure defining a freespace or ceiling void 8 which is used as a technical service zone forcables, light fittings etc.

Open racks 9 or rack cabinets, into which electronic devices may befitted, are disposed on the raised floor 2 within the ‘technical space’defined by the room 1. Power and data cables for the racks 9 may runthrough the floor void 6 of the raised floor, the ceiling void 8 of theceiling structure, the room space 1 over the floor or beneath theceiling, or any combination of these. When cables are run at a highlevel in the room space 1, the suspended ceiling 7 is often omitted.Often, so-called static cables should be located in the floor void,these cables being mainly heavy copper cable such as for power supplies,and control data cables, all of which may be expected to remain in situfor extended periods. Conversely, more fragile or frequently-movedcables should be located at a high level within the technical space orin the ceiling void 8 where they can be concealed by removable panelsand/or supported by suitable supports (not shown). These cables mayinclude data cables such as fiber, twisted-pair and coaxial cables. Suchcables are relatively vulnerable to damage, as may for example be causedby maintenance engineers walking on exposed cables when the floor panelsare lifted.

Positioning such cables at a high level reduces the risk of damage andeases access to them for installation, maintenance and re-routing.

A rack 9 comprises a vertical framework of rails provided with suitablemounting holes to appropriate industry standards (e.g. EIA-310-D), toaccept electronic equipment. The rack 9 is either open to the room space1 or is mounted inside an enclosure to form a rack cabinet 11 which hasvarious air inlets and air outlets allowing cooling airflow to reach theelectronic equipment 12 and to carry heat away.

Electronic equipment most commonly ventilates front to back, that is,air flows through ventilation holes in the front of the equipment casingand exhausts through holes in the rear of the casing. Small fans,usually in the rear of the casing but in some cases in the front, assistthis through airflow. The heat from the electronic components within thecasing is dissipated by convection or individual fan assistance intothis through airflow, thus effecting cooling of the equipment. There aresome items of equipment that ventilate bottom to top, or side to side,but the principle remains the same.

The rack cabinet 11 will most often be provided with ventilation slotsor perforations in the front and rear doors to provide for throughairflow. Many rack cabinets will also have a ventilation inlet within acabinet bottom plate 13 and another in a cabinet top plate 14 to avoid aconcentrated heat build-up in the top of the cabinet due to theso-called ‘stack effect’. Either of these two openings or in some casesboth may be augmented by fans 15, preferably in multiple arrays toprovide for redundancy.

The raised floor system 2 comprises a plurality of floor panels, some ofwhich are solid and some of which are perforated or of a grille-typeconstruction 2 b permitting airflow through them from the floor void 6to the room space 1. The suspended floor panels are supported onpedestals 2 a which are fixed to the base floor 5 by screws and adhesiveon a regular modular grid. The airflow from the perforated panels orgrilles 2 b flows out into the room space 1 and is drawn through thefronts of the equipment cabinets 11 into, through and between the unitsof electronic equipment within.

Sometimes dampers will be fitted to the floor grilles 2 b to allowadjustment to the airflow and manual balancing of the room loads.

At the perimeter of the room 1, a plurality of close control computerroom air conditioning units 16 (CRAC units) receive air flowingperpendicularly to the rows of rack cabinets 11. In large rooms, some ofthese CRAC units 16 may also be provided within the room space away fromits periphery to overcome distance limitations upon the effectiveness ofthe units 16. Also, CRAC units 16 may sometimes be positioned outside aroom and connected to it through appropriate openings in the perimeterwall of the room 1.

Each CRAC unit 16 comprises a heat exchanger or coil 17 and a fan 18.Exhaust air from the rack cabinets 11, mixed with room air, is drawninto an inlet 20 of the CRAC unit 16, across the cooling coil 17 andthrough the fan 18 and out an outlet 19 into the floor void 6. There arevarious types of CRAC units each of which rejects the room heat absorbedin different ways, namely chilled water units, direct expansionwater-cooled units, direct expansion air-cooled units, direct expansionglycol-cooled units and others. However they typically share the sameprinciple, which is that the absorbed heat is conveyed to a suitablepoint outside the room 1 where it is dissipated to atmosphere usingconventional air conditioning technology. Thus, the unit is connected toa central plant via a cooling circuit which may consist of distilledwater or other refrigerant. The cooling circuit dissipates heat to theatmosphere outside the technical space via heat exchangers such ascooling towers or external radiators (dry coolers).

The CRAC units 16 described and illustrated are referred to as‘downflow’ units reflecting the generally downward airflow within themin use, but ‘upflow’ units are also available. Upflow CRAC units areused, for example, where there is insufficient room height for asuspended raised floor or where the equipment servicing philosophy issuch that services are run overhead, thus obviating a raised floor.Either way, the principle is similar except that in the case of anupflow CRAC unit, the air inlet is at the front bottom of the unit androom air passes up through the unit before being expelled at the topwhere it moves out and down in front of the rack cabinets 11 beforebeing drawn through the electronic equipment as in the previousscenario.

Ambient room air is typically at a temperature of 22° C.±2° C. withrelative humidity of 50%±5%. The CRAC units 16 supply chilled air to thefloor void 6 at approximately 13° C. which is drawn into the rackcabinets 11 by either convection from the airflow from the perforatedpanels/grilles 2 b or by the effect of the cabinet fans. As the chilledair passes through and around the electronic equipment units and isheated, it exhausts out into the room space 1 at approximately 35° C.The heated air then mixes with the conditioned ambient air of the roomwhich is at a temperature of approximately 22° C., and the mixed airthen returns to the CRAC units 16 at a temperature of approximately 24°C.

Other perforated tiles 2 b are positioned throughout the room 1 toprovide air supply to other heat-generating equipment and to maintain anappropriate ambient environment in the room 1. Some rooms are laid outon the principle of ‘hot and cold aisles’, in which rows of cabinets 11are arranged so that their fronts face each other across a ‘cold’ aisle,from which cold air is drawn into the opposed cabinet fronts, and theirbacks face each other across a ‘hot’ aisle, into which warm air isexhausted from the opposed cabinet backs. Perforated panels or grilles 2b are only placed in the cold aisles (other than those serving otherpieces of equipment elsewhere in the room), thus seeking to ensure themaximum cooling effect by not mixing cold and hot airflows in the sameaisles.

The large floor-mounted CRAC units distributed around the perimeter andsometimes in the center of the technical space take up floor space androom volume that could otherwise be devoted to racks or cabinets. Thisultimately costs money by loss of potential revenue generation. However,the alternative of air conditioning vents positioned in the ceiling orin walls close to the ceiling, as commonly found in offices, is notsuitable for a data center. This is because as heated air rises from theserver racks or cabinets, it mixes with the cooler air blowing down fromthe air conditioning vents leading to condensation and formation ofwater droplets which can damage computer equipment. Hence, it ispreferred that air conditioning vents are located below the server racksso that the natural airflow is not disrupted.

As can be seen in FIG. 1, the cold air cooled by the condenser is forcedout 5 underneath the CRAC unit 16 below a raised floor 2 on which theCRAC unit 16 is mounted. The raised floor 2 acts as a plenum underpositive pressure. Some of the cold air is forced up throughcable/ventilation holes in the base of the equipment cabinets 11 mountedon the floor, while some rises through floor ventilation grilles 2 baround the cabinets fitted with control dampers. Thus, the total volumeof the air in the technical space is conditioned.

As already mentioned, some of the conditioned air in the technical spaceis drawn through the equipment cabinet 11, for example via perforateddoors, by small fans within the servers themselves. This air flowsthrough and around the heat-generating electronic components within theservers and exhausts as hot air at the rear of the server into thecabinet 11. In some cases, however, solid doors are used at the rear ofthe cabinet 11 and hot air is expelled at the top of the cabinet 11through an opening, sometimes assisted by additional fans 15 to avoid aconcentrated heat build-up in the top of the cabinet 11 due to the‘stack effect’. The hot air then returns to the room 1 where it mixeswith the room air and is eventually recirculated through the CRAC unit16 from which its heat is ultimately rejected to the atmosphere viasuitable heat transfer means as aforesaid.

With a general environmental controlling device as described above, allthe air within the technical space is being continuously treated.Unsurprisingly, the energy demands associated with such an approachrepresent a significant cost factor. Also, the cooling of individualservers relies heavily upon their internal fans and there may be noattempt to ensure that each server receives its necessary share ofconditioned air. Instead, conditioned air may be introduced into thecabinets 11 by various imprecise means that can give rise to conflictingairflows.

Once in a cabinet, conditioned air is left to flow within the cabinet ina way that depends upon the disposition of equipment within the cabinet.So, for example, a server might receive inadequate cooling becauseadjacent servers nearer the air intake take a disproportionate amount ofconditioned air. Similarly, conditioned air might bypass a server byfollowing a path of less resistance, for example through an adjacentempty equipment bay within the cabinet 11. Also, if a server fan shouldfail, that server will almost inevitably overheat.

The floor void 6, when used for delivery of the cooling supply air, isoften assumed to consist of an even mass of pressurized air deliveredfrom a number of CRAC units 16, arranged around the perimeter andpossibly the interior of the technical space. The reality is that thefloor void 6 contains a plurality of independent airflow plumesemanating from each CRAC unit 16, segregated by boundary layers. Each ofthese airflow plumes varies in size due to other factors which affectsthe amount of cooling which can be provided to the technical space.

A factor in airflow plume development is the static pressure within thefloor void 6. Assuming initial design is correct, a lack of staticpressure may arise from poorly-managed floor openings and/or fromclose-coupled rack cabinets with additional fans. Specifically, cut-outsfor cable entry below cabinets and elsewhere within the room, togetherwith excessive perforated floor panels or grilles, causes overcooling,loss of static pressure and wasted capacity. High-pressure areas of thefloor are overcooled while low-pressure areas overheat because a loss ofstatic pressure reduces the size of a plume and hence the volume of roomspace that that plume is able to cool.

To mitigate this effect, hole cut-outs should be sealed around cablesand the floor grilles 2 b should be adjusted to deliver an accurateamount of airflow to each cabinet 11. However, in practice,poorly-fitting floor panels or, more usually, floor panels that havebeen lifted and replaced badly can result in substantial leakage ofcooled airflow from the floor void. If the floor void is used forcontaining cabling then engineers installing cables typically remove acomplete row of floor panels and/or stringers rather than leaveoccasional panels (typically every fifth panel) in place to keep thefloor ‘locked-in’, with the result that the panels shift across thewhole floor in a process called ‘fish-tailing’, causing gaps to open up.

By way of illustration, site investigations have verified cases whereonly 31% of the total cooling airflow was being distributed through‘engineered’ openings, with the remaining 69% circulating out of cablecuts, gaps around equipment and openings from rack cabinets. The coldair escaping in this way returns to the CRAC units 16 withouteffectively transferring heat from the equipment. This cold ‘return’ or‘bypass’ air disrupts the heat transfer that could have been availableto overloaded air conditioners, in such cases reducing the effectivenessof the CRAC units 16 to just 52% of their capacity.

The act of installing cabling within the floor void 6 further restrictsairflow through the floor void 6. This is a degenerative effect, as therack cabinets 11 are populated over time and a potentially greater heatload created, the additional associated cabling further restricts theairway supplying the cooling airflow through the floor void. Thecombination of new cabling technology, in which cables tend to be oflarger diameter, together with electronic equipment ‘compaction’ resultsin more densely occupied equipment spaces, connected by increasingamounts of cable.

While it is correct in principle to attempt to reduce the inlet airtemperature entering the rack cabinets 11 by increasing the airflow ratethrough the perforated floor panels, this is an oversimplification. Ifthe velocity is too high, then the airflow can overshoot a rack cabinet11, tumbling into the hot aisle at the rear of the rack cabinet 11. Thiswastes supplied chilled air and, by mixing with the exhausted heated airfrom the rack cabinet 11, lowers the temperature of the exhausted airand therefore reduces the capacity of the installed air conditioners.

Moreover, unless carefully engineered, increased air velocity can createa ‘wind tunnel’ under a raised floor. The increased air velocity reducespotential static pressure, and may be so high that sufficient staticpressure to deliver adequate volumes of cooling air up through the floormay not develop for 9 m to 12 m beyond the point of fan discharge fromthe CRAC unit 16. This results in insufficient static pressure close tothe CRAC unit 16 to move the available cooling air up through the floorgrilles 2 b. Worse still, in some cases, heated room air is actuallysucked down into the floor void through the grilles 2 b, reducing thecooling capacity of the cooling airflow and creating ‘hot spots’.

The objective of hot and cold aisles is to separate the source ofcooling air from the hot air discharge which returns to the CRAC unitinlet. However, in practice, such physical separation is difficult toachieve in an open room environment particularly where high heat loadsare concerned. Close-coupled rack cabinets each provided with extractfans create a ‘chimney’ effect to pull air from the raised floor upthrough the cabinet and the equipment therein. Too often, however, thesefans exhaust more air than the CRAC units 16 can deliver, therebyoverwhelming their cooling capacity. Also, excessive suction created bythese rack cabinets 11 causes heated air from the room 1 to be pulledinto the floor void 6 and then up into the rack cabinets 11. There isjust not enough cold air from the CRAC units 16 to satisfy theoverwhelming quantity of air exhausted by the rack cabinet fans. Somestudies have revealed that bypass air problems typically limit CRACunits to less than 35% of their ‘nameplate’ rating. In ‘hosted’environments, close-coupled rack cabinets have earned the title ‘badneighbor devices’, in that they take more than their share of theavailable cooling airflow.

Orientation of the CRAC units 16 in relation to the rows of rackcabinets 11 is not significant at low loads with a clear floor void 6.However, as cooling loads or cabling and other sub-floor obstructionsincrease, their orientation becomes significant. Ideally, the CRAC units16 should be orientated such that their airflow is perpendicular to therows of rack cabinets 11, as placing them parallel to the rows of rackcabinets 11 will tend to create hot spots. This orientation of the CRACunits 16 may place an ultimate limit on total cooling capacity. Forexample, two out of four wall surfaces in a room may be available forlocating the CRAC units 16, the longest of which are approximately 2.4 mwide, with a capacity of approximately 100 kW. Placing more CRAC units16 on the other two walls will almost certainly result in disruptedairflow/turbulence. More CRAC units 16 can be added within the body ofthe technical space (which will typically be the case in wide datarooms) but this inhibits data rack layout flexibility.

Humidity should be maintained at a level that avoids static electricityproblems. However, to provide stable humidity, it is not advisable toequip each CRAC unit 16 with a humidifier. Slight drift in humiditysensor calibration may cause a CRAC unit 16 to add humidity while anadjacent CRAC unit 16 is simultaneously trying to dry out the air.

This fails to provide a stable environment and pours significant energydown the condensate drain, increasing risk, maintenance, repair andcapital costs. Rather, good practice suggests that a centralized systemfor humidification should be used, which is usually the make-up airsystem for the room space. If the chilled water temperatures are toolow, this shifts cooling coil performance toward dehumidification andlowers cooling capacity.

Those skilled in the art will appreciate that individual CRAC unitscannot share load with their adjacent or opposing partners and in mostsituations the temperature gradient varies widely due to the variety andcapacity of the items of heat-generating equipment as well as theiroperational state at any given moment in time.

Once cooled air has been delivered through the floor void 6 and into theroom space 1, that airflow enters the rack cabinet to cool the equipmenthoused therein. Conventional rack cabinets have perforated doors frontand back to allow through airflow front to back. This through airflow isachieved by the combined action of: air being drawn through by smallfans associated with the equipment itself; air being drawn through byfans associated with the cabinet (e.g. mounted at the top, bottom,middle etc.), if fitted; forced convection from the raised floorperforated panels or grilles; and/or forced convection directly into thebottom of the cabinet.

Perforated doors may work satisfactorily at relatively low heat loadsbut, with high density loads, the doors themselves offer resistance tothe desired through flow. While cabinet fans can help to eliminate thehot spots that tend to occur at the top of the rack cabinet 11, care hasto be taken in sizing these fans in relation to the through airflow.Tests have shown that the cabinet fans can set up a strong ‘chimneyeffect’ airflow pulling the air out of the top of the cabinet 11. Thisprimary airflow entrains the room air at its boundary, increasing themass of moving air while reducing its velocity. This tends to set upsecondary circulation and reduce the through flow into the equipmentitself.

Mention has already been made of the problem with air from theperforated floor panels or grilles being of such a velocity as to passover the cabinet 11 and into the hot aisle. Conversely, lack of velocitycan result in a cooled air supply stopping less than half way up thecabinet 11 and therefore not reaching equipment at higher levels. Thisequipment will rely on cooling provided by the room air which is drawnthrough by the equipment fans, which room air may itself already beheated and of limited cooling capacity.

Forced convection directly through the bottom of the cabinet 11 mayresult in similar problems to those noted above. However, additionally,if the rack 9 is heavily populated, then the incoming air strikes thebase of the first server and may be deflected out of the cabinet 11through the perforated doors both front and back. This wasted cooled airthen mixes with the room air. During tests using standard industry rackcabinets with forced ventilation through the cabinet base full of 1Uhigh density servers, it was found that the temperature gradient at therear of the rack became inverted with the highest temperatures recordedwithin 150 mm of the base of the cabinet. This was largely due to theincoming chilled airflow being forced directly out of the cabinet by thelowest servers and back into the room 1.

In general, conventional rack cabinets are rather ‘leaky’ not justexternally but also internally: for example, many have gaps between therack itself and the cabinet enclosure allowing cooled air to bypass theequipment within the cabinet and be wasted.

Currently, good practice dictates that due account is taken ofindividual cooling requirements when arranging the deployment in a rackcabinet, especially to avoid placing very hot devices below equipmentwith lighter heat loads. Even where this practice is followed, risingheat will tend to result in an accumulative heat build-up progressivelytowards the top of the rack cabinet.

The majority of rack cabinets are never full of hot devices, up to 40%percent occupation density being typical. Sometimes relatively highloads are possible within standard cabinets—perhaps up to 5 kW. However,upon examination, this is usually due to the load being created byrelatively few devices. For example 5 kW of aggregate cooling load fromtwo items of equipment with plenty of air space between them is verydifferent from 5 kW of aggregate cooling load from a full rack of hotequipment. Also, equipping the server itself affects the resistance toairflow. For example, a server fully equipped with network cards mightoffer 64 pascals of resistance while an otherwise identical but lessequipped server might offer only 20 to 30 pascals of resistance.Further, the loads of adjacent equipment directly impacts coolingcapacity.

If one considers that a raised floor is effectively a supply air ductand that the length of a typical data room is considered as the ductwidth (say 37 m) and a room area of approximately 1000 m2 (27 m×37 m) istaken as an example, then taking the averaged loads across the room:

-   -   (a) for a floor void height of 600 mm, and a heat density of        2000 W/m², the duct should be 108.9% of the room length;    -   (b) increasing the floor void height to 800 mm for the same heat        density results in the duct being 81.9% of the room length; and    -   (c) if the floor void is cabled out, reducing its effective        depth to only 300 mm, then the duct should be 218.1% of the room        length. Put another way, the maximum achievable heat density is        less than 2000 W/m² (108.9%).

Further if one considers the maximum floor space that can be occupied byactive IT hardware (rack cabinets) in the above scenario (typicallybetween 30 and 35 percent allowing for all white space such as serviceclearance, access aisles etc.) and a rack cabinet footprint of 0.54 m²(0.6 m×0.9 m), then each cabinet can provide between 3.6 kW and 3 kW ofcooling capacity. This is a theoretical capacity with a completely freefloor void: more realistically, with cabling in-situ, this figure willdrop to between 1.8 kW and 1.5 kW per cabinet.

Using the same scenario, consider the total available cooling load fromcorrectly oriented CRAC units. Assuming access and fire exit doors areallowed for (one per side, 1.2 m wide) then a maximum of 14 CRAC unitsof 100 kW capacity can be located on each side. Assuming a minimumavailability of n+1, then a total of 26 CRAC units are available for aload of 2,600 kW. This equates to 2.6 kW/m² or 4.68 kW to 4 kW ofcooling capacity per cabinet. However, to be able to deliver thiscapacity, the size of the supply duct would need to be increased byraising the floor void height to 1500 mm of clear space (i.e. above anycabling also within the floor void, which is impractical for most datacenters. Even for new purpose-built facilities, this floor depthpresents various technical challenges.

Other factors affecting the movement of airflow within a given space areceiling and wall topography, such as: surface characteristics; type ofsurface; downstand beams; surface obstructions; pipework; ductwork;services; wall abutment; placement of supply grilles relative toequipment racks; placement of equipment racks relative to each other;other equipment; and wall abutments.

Predicting the effects of these various parameters with any certainty toachieve an optimized room configuration is extremely difficult.Technology such as computational fluid dynamics (CFD) software canassist greatly but this approach is not, as yet, widely adopted withinthe industry. Also, to be effective, the computational model requiresaccurate modelling of all the characteristics of the room, the rackcabinets and the heat producing devices. Many manufacturers do not makeavailable the information necessary to undertake this task.

The result is that while some security disadvantages have been overcomeby containment devices which contain electrical equipment and arepositioned in an environmentally controlled room, many of these existinginstallations risk overheating the equipment due to unexpectedly denselevels of deployment and poor or inadequate air extraction andventilation. Indeed, in some data centers, the doors that are supposedto be closed to provide security are instead left open to assistcooling.

Enclosed cabinets with perforated ceilings onto which fan kits can beattached to aid air circulation through the units are also known, butthey are ineffectual in high density applications. These systems stillrely on environmentally cooled room air and the cooling effect of thefans is negligible. While fans may help to achieve desirably uniform airflow within the cabinet, considerable care should be taken in specifyingfan size and capacity because an incorrectly specified fan can inhibitthe general flow of air within the cabinet.

Various solutions have been proposed to assist with issues ofconventional data center technology, which is now some thirty to fortyyears old. These break down into two main approaches: room-levelsolutions and rack-level solutions.

Room-level solutions begin with ‘close coupled’ cabinet systems.Essentially these are developments of the existing pressurized floortechnology. The individual cabinets attempt to make better use of thesupply air by controlling the amount of air entering each unit with avariety of different types of air inlets or dampers. These dampers arepositioned in the base of the cabinet, and on the cabinet front,normally low down to take advantage of the coolest level of room air.Some have pre-engineered openings for cables with sealing brush stripsto mitigate the effects of unmanaged and unsealed cable cut-outs. Often,bottom air inlets are provided with small fans to assist airflow intothe cabinet enclosure, and in some cases fans are also provided in thetop of the cabinet enclosure for the same purpose.

Some variants have recognized the limitation on total cooled air supplyfrom the floor void i.e. between 2 kW/m² and 3 kW/m² maximum. Thesevariants attempt to use the room air in addition to that from the floorvoid to increase the total cooling capacity, which is achieved bymanually adjustable opening vents in the cabinet front.

Existing room-level solutions do nothing to address several issues withraised floor technology, which include:

-   -   limited total cooling capacity with reasonable floor void        heights;    -   increasing sub-floor obstructions reducing the airway over time;    -   wasted cooling air through poorly-managed cable cut-outs and        leaking floors and cabinets;    -   lack of or too high a static pressure;    -   uneven loading on CRAC units with some under-utilization and        some overloaded;    -   problems of bypass air;    -   uneven temperature gradients across the room space; and    -   influence of adjacent items of equipment on each other.

Further, in attempting to boost the cooling capacity by using room air,these solutions assume that a background ambient temperature of around22° C. is available. For the reasons already given, this is often notthe case as room air is heated by recirculated exhaust air. Using fansto pull air through a cabinet requires a careful balance and sizing offans to achieve the desired through flow with the racked equipment.Strong vertical airflows can be set up which remove much of the cooledair before it reaches the equipment and has the effect ofthrottling-down the through flow desired and being attempted by thesmall racked equipment fans. The vents and dampers on these systemsnormally require manual adjustment which is carried out on atrial-and-error basis. As racks are equipped out over time, adjustmentsare often not made to the damper settings until there is a problem.Perhaps more seriously, close-coupled systems can draw so much air fromthe floor void that they exceed the capacity of the CRAC units to supplyit, and starve other containment systems of cooling in a prime exampleof the aforementioned ‘bad neighbor devices’.

A different approach is used for ‘spot coolers’ which, in one example,places a heat exchanger on the rear door of the cabinet, complete with anumber of fans. The fans pull room air through the cabinet and theequipment racked therein, into the heat exchanger and then exhaust itback into the room at or near room temperature. The airflow is thereforefront-to-back through the racked equipment as required by the majorityof equipment suppliers.

The heat exchanger coil is connected via a pipework system to a coolingdistribution unit located outside the technical space which, byregulating the temperature and flow of the chilled water in relation tochanges in room dew point, helps prevent condensation. The coolingdistribution unit is connected to the existing chilled water supply ofthe building. Between twelve and fifteen heat exchangers can becontrolled from one cooling distribution unit, giving a cooling capacityof up to 8 kW per cabinet.

This system has been designed as a ‘hot spot cooler’ for retrofitsituations. The heat exchanger used, however, intrudes into the hotaisles by 150 mm, thereby reducing the width of each hot aisle by 300 mmif used in adjacent opposed rows.

Due to uneven temperature gradients and complicated airflow patternsalready discussed, it is a distinct possibility in many data rooms thatthe room air drawn into the rack cabinet is above the designed ambienttemperature, e.g. 22° C. In that case, the air being exhausted back intothe room by the cabinet heat exchanger may also be above the designedambient temperature. In the case of hot aisles where ambient air is at,say, 34° C. (assuming other containment is exhausting into them—thesituation one would expect with a retrofit hot spot solution), therewill be some cooling of the air in the hot aisle by mixing of cooler airfrom the cabinet heat exchanger. This may lead to a phenomenon called‘static bypass’ which lowers the cooling effect of the CRAC units,creating other hot spots.

While connections are made into the existing building chilled waterservice, which would not be permitted in some data facilities, thissystem has the advantage of allowing progressive build-out. Providedthat care is exercised in positioning relative to other equipment, itprovides a good technical resolution over conventional technology withregard to equipment cooling, albeit limited given the heat loads nowbeing encountered. However, the system is ‘open loop’ and so is stillvulnerable due to complicated airflow patterns within a conventionalsystem. Similarly, the equipment in the rack is exposed to the otherissues already discussed with conventional open rack cabinet systems,such as dust, moisture, cold smoke damage, security and fire risk.

Moreover, the cooling distribution unit is linked to remote room-mountedtemperature and humidity sensors. So, this is a ‘centralized’ controlsystem rather than a rack-specific control system.

The pipeline supply system connecting the cooling distribution unit tothe rack heat exchanger is typically single pipe with a mechanicalcoupling joining the pipe sections. The type of coupling used is a‘Victaulic style 606’ (trade mark) which provides a very high qualityjoint. However, such a joint cannot be said to be leakproof, andcombined with the use of solenoid valve assemblies in the pipe runs, asvalves are a potential source of leaks, the pipe system may have leaks,even if dual-piped which is not a standard option. If a leak isdetected, an internal purge system pumps the coolant within the coolantdistribution unit to a drain.

Another variant of the ‘open loop’ system is the ‘zero floor space’model. One embodiment of this approach locates a heat exchanger at highlevel above the rack cabinets such that cooled air is washed down thefronts of the individual cabinets. This is a similar action to that ofthe pressurized floor solution, but in the reverse direction. Theairflow passes through the racked equipment due to the action of theinternal equipment fans and forced convection from the overhead heatexchanger fans. The exhausted heated air is then drawn back up into theoverhead heat exchanger to be cooled and the cycle repeats.

The overhead heat exchanger is connected via a pipework system to acooling distribution unit located outside the technical space and thento the building's chilled water supply. Once again, this system relieson central control using a remote room temperature/humidity sensor.

Each module of the system is 1.83 m×1.8 m in plan area which coversthree conventional rack cabinets and weighs 160 kg when filled withcoolant. The units are attached to the structural soffit by threaded rodand appropriate anchors, which means that this solution is not, at leastprimarily, a retrofit option but for new-build situations.

Each module is 0.55 m high and has between 0.6 m and 0.9 m of clearspace between the module's bottom face and the top of the rack cabinets.Any suspended ceiling fitted is located at the same level as the coolingmodule. The spacing between the modules in plan is varied to suit theroom load. If the units were butted together edge-to-edge, this wouldgive a notional 6.6 kW per cabinet of cooling. However, this level wouldnot be achieved in reality as the system is effectively ‘open’ andsubject to all the same room restrictions as for a pressurized raisedfloor. With edge-to-edge abutment or substantially so, there isinsufficient room to install light fittings or overhead cable management(which is increasingly the preferred option among users), either ofwhich would disrupt the airflow pattern if fitted below the coolingmodules.

Placing the units edge-to-edge on their shortest sides (to fitcabinet/aisle widths) and, say, two cabinets apart on their longest sidewould provide a notional 4 kW per cabinet. The real cooling loaddelivered to the rack cabinets is likely to be just above that providedby a pressurized raised floor. However there is the advantage that theoverhead situation does not have to deal with the floor void and roomtopography restrictions to airflow inherent in the raised floor design.The supply and return air path from rack cabinet to heat exchanger isrelatively short, and floor space is saved for use by revenue-generatingequipment. The raised floor can be used for static cabling and possiblydynamic cabling can be run overhead although even with spaced-apartmodules, cabling along the line of the rack cabinet is not possible at ahigh level other than directly on top of the cabinets. Nevertheless thespaces between the cooling modules could be used for cable bridgesbetween rows.

The next category of solution is the closed-loop chilled water groupwhich is sometimes described as ‘air cooled’ systems because only air isused within the rack cabinet itself However these systems are, inreality, a development of the traditional pressurized raised floortechnology, in that they rely on CRAC units to transfer the heat fromair to water or refrigerant and then ultimately to the atmosphere viaexternal chillers or water towers.

One particular example of this approach, as disclosed in InternationalPatent Application No. WO 01/62060, seals the cabinet and directs theairflow via a front and rear plenum or manifold. Actually the cabinetconstruction may not truly be sealed because in practice there may bevisible gaps in the carcass construction, although there are sealinggaskets on the doors. The air movement is vertical through the front‘supply’ plenum, then horizontal through the racked devices and thenvertical again through the rear ‘exhaust’ plenum. A variety of fans areused, sometimes located at the top of the exhaust plenum and sometimesalso at the bottom of the supply plenum. This helps to control theairflow through the racked equipment.

The bottom of the supply plenum is connected into the raised floor,which effectively forms the supply duct. The top of the exhaust plenumis connected into the suspended ceiling void which effectively fauns thereturn air duct. To achieve the function as discrete ducts, the twovoids are segregated by vertical barriers. This arrangement allows for asubstantial improvement over the normal pressurized raised floor openroom return air scenario. A limited number of rack cabinets are directlyconnected to individual CRAC units forming a closed loop system, thusmaking far more efficient use of the cooling air available from the CRACunit. The supply air is delivered at 13° C. while the return air isexpected to be 34° C. to 35° C. This compares with a conventionalpressurized floor open room scenario of supply air at 13° C. and returnair between 22° C. and 24° C. Thus, it can be seen that the closed loopsystem has a Δt of 22° C. as opposed to the conventional system Δt of11° C. The principal advantage put forward for this system is that bydoubling it, it is possible to reduce the required airflow of cooled airto the rack cabinets. This in turn means the CRAC unit fan requires 50%less power to drive the airflow and thus substantial energy savings arepossible. However, this figure assumes that system losses do not reducethis saving even though it is still proposed to use the raised floor forcabling, and other factors such as leakage through the floor tiles.Similarly, a suspended ceiling is not a largely unobstructed ductnormally used for ducted air supplies so, again, resistance to airflowand leakage within and from the ceiling is likely to impact on thesefigures.

Also, with this approach, in a retrofit situation, it may not bephysically possible to install a suspended ceiling due to the amount ofoverhead service obstructions already existing. Additionally, installinga suspended ceiling in a live data center may not be acceptable i.e.drilling into the structural soffit to fix the suspension hangers and soon.

This system still has stack or chimney effects inherent to all verticalairflow systems, requiring very careful management of the deployment ofheat-generating devices. While loads of up to 8 kW of cooling areclaimed for this solution, this may be difficult to achieve in practice,even assuming the greater fundamental efficiency due to the higher At.Test figures have apparently been based on equipment nameplate ratingsor alternatively using heater bars. Real figures can be a third ofnameplate ratings under real running conditions. Also, while the use ofheater bars is the most common industry approach to testing, this takesno account of the variation of heat dissipation across electronicdevices or their resistance to airflow—typically 20 pascals for anear-empty server or other device and up to 64 pascals for one full ofnetwork cards. Fully-ducted systems of this type utilizing discreteductwork and well-sealed cabinets can achieve cooling loads of up to12.5 kW. However there are limitations on the depth or length of cabinetrows due to air velocity factors—testing has indicated this to be ataround 20 standard cabinets (600 mm wide).

Another issue with of this approach from a user's point of view relatesto the lower airflow. Original Equipment Manufacturers (OEMs) designtheir equipment such that small fans sometimes combined with heat sinksmove the heat away from the critical components and up into the throughair-stream. Additional fans, sometimes as many as eight, pull airthrough the device to exhaust the heated air at its rear. An OEM'sproducts can be damaged by too high an airflow; especially, it ispossible to ‘windmill’ or cycle the small fans beyond their self-drivenrevolutions and shorten their life or indeed burn them out prematurely.Overcooling can also prejudice the correct operation of a device. On theother hand, too low an airflow can result in local overheating.

The heat levels across a device are not even, some regions beingsignificantly hotter than others. Each manufacturer has varying inlettemperatures for its equipment. However, in most real data centersituations there is a mixture of products from different suppliers, ordifferent models from the same supplier in any given rack. Therefore,from a practical viewpoint, a compromise airflow is provided that coversthe spread of inlet temperature requirements and variations experiencedacross the devices. For this reason, the major OEMs have expressedconcern regarding any cooling method which intentionally reduces theairflow significantly. Their preference is to tend towards higherairflows as this is more likely to ensure safe operation, rather thanmove to lower flows.

As with some of the other systems discussed, the control function onthese products is effected centrally. The cabinet enclosure is, asalready noted, leaky especially if the enclosure is a single-skinconstruction, especially if not insulated. Consequently, should it bepossible to achieve real cooling loads above 5 kW and perhaps up to 8 kWwith this equipment, then there will be an impact on adjacent equipment.In general, other neighbouring hot devices are likely to impact on theenvironment in a given cabinet. While this system includes door seals,the overall cabinet construction does not appear to meet any recognizedstandard of ‘sealing’ classification.

The next group of products fall under the generic grouping of ‘sealedclosed loop air to water category self-contained’. Put simply, the heatexchanger is contained within the rack cabinet itself. Embodiments ofthe disclosure fall within this category but at least one other exampleis also currently within the market. This unit, the subject of U.S. Pat.No. 6,506,111 issued Jan. 14, 2003, has a segregated supply and exhaustairflow system comprising two plenums—one at the front of the rackedequipment and the other at the rear. This is in common with other unitsdiscussed previously.

The whole unit stands on a plinth which contains the heat exchanger coiland fans. Heated air from the rear of the racked equipment is drawndownwards through the plenum and into the plinth. The fans push the airthrough the heat exchanger coil and up into the plenum in front of theracked equipment, through the equipment and back into the rear plenum tobegin the cycle over again.

To overcome the stack or chimney effect inherent in any vertical airflowsystem, various distribution devices are incorporated into the frontplenum. The first of these comprises a fiat plate containing a pluralityof pre-formed regular apertures.

These apertures increase in number from the bottom to the top of theplenum and thus allow the air to flow through them into the rackedequipment. The pattern of the holes can be varied to suit the load withthe intention of delivering ‘approximately’ equal airflow to all levelsof the racked equipment. It is also possible to have louvres fitted tothe apparatus to provide further airflow adjustment, presumably in amanual operation.

Another distribution device option comprises a solid panel with a raisedside along its two long edges and a curved roll-overtop mounted on theback of the front door of the rack. The panel is tapered in its depthsuch that it reduces the cross sectional area of the plenumprogressively from its bottom to its top. This again is designed toprovide even airflow through the racked equipment, in a manner similarto the progressively-reducing section found on any run of heating,ventilating and air-conditioning (HVAC) ductwork. Further it is proposedthat the same device can be fitted in the rear exhaust plenum or in bothplenums. How successful this system is in providing even airflow acrossall racked equipment is not known, but it would seem that the currentlevels of cabling required might be obstructed by these devices inhigh-density applications.

Multiple fans are provided for redundancy although it appears that it isnecessary to 25 withdraw the complete fan tray to replace a failed fan,with the consequence that airflow is interrupted while this takes place.With high-density applications, even a short period while a fan isswapped out could have serious implications for the racked equipment.Similarly the heat exchanger coil is described as either single, whichis not resilient to the minimum requirement of most data centers for‘n+1’ unless hot swappable, or multiple. The multiple option wouldprovide resilience although if it is necessary to pull the two coils outtogether to swap out the defective one there does not seem to be anypoint in a multiple coil arrangement. Perhaps this is the reason thatthe production units are only equipped with a single coil.

The system of U.S. Pat. No. 6,506,111 has the benefit of removing heatfrom the racked devices close to where it is generated, and of directingthe airflow. This also allows relatively high heat loads to be dealtwith—currently up to 10 kW per cabinet is claimed. A raised data roomfloor is not necessary and the position of the heat exchanger means thatsmall footprint dimensions are achieved, albeit with a loss in rackheight (the current variant is 40U). The items containing coolant arelocated low down in the plinth reducing potential damage from leaks,although no means of leak containment seem to be provided. Althoughdescribed as a sealed system, this refers to ‘close fitting’ panels'(single skin non-insulated) and not a recognized seal standard orrating—thus the cabinet can only be used in data room environments andis subject to dust, cold smoke, water etc. penetration. Externalchillers and interconnecting pipework are required.

The final group of known products are sealed closed loop air torefrigerant systems which exhaust the extracted heat into thesurrounding space. These systems are substantially autonomous, but mostsuitable for use in low (or at least not high) density environmentswhere the hot exhaust air will not add to problems for other equipment.However, although some of these products are sealed to a recognizedstandard, they do not have any means for safeguarding the internalenvironment where the external environment is not benign, such asconditions of high humidity, partial pressure problems and so on. Somemodels have the self-contained package unit mounted externally on top,or on the sides of the cabinet. Other variants have it located in thebottom of the rackable area. Cooling capacity tends to be limited withthese products, ranging from 1.5 kW up to 4 kW. A condensate drain isrequired with these products; excessive opening of doors or poor sealscan cause continuous drainage of condensate. The room in which theseproducts are located needs to have adequate air circulation to ensurethe heat exhaust is rejected to avoid the cabinet overheating.

Examples of such cabinets are currently sold by Liebert Corporationunder the name ‘Foundation’ and by Stulz GmbH under the name ‘CT CoolingRack’. All trade marks are acknowledged.

Liebert's ‘Foundation’ is aimed at small offices rather than datacenters. Essentially, it is an enclosed cabinet, which may be sealed,with locks on the outside of the cabinet to prevent tampering. Aninternal rack-mounted UPS is an option. Various cooling modules can beemployed, for example an internal rack-mounted or external top-mounted‘environmental control module’ that cools equipment within the cabinetby using ambient air to remove heat from the inside of the cabinetthrough an air-cooled condenser. This of course takes some of the spacethat could otherwise be devoted to servers, if their heat generationproblems could be overcome. Warm air is exhausted near the bottom of theunit.

Other cooling options are a fan that can be mounted inside the cabinetto promote air circulation in the cabinet, and a back-up cooling modulewhich responds to excessive internal temperature by circulating filteredambient air through the cabinet. Another cooling option is aceiling-mounted fan for ventilating a confined space outside thecabinet, heated by warm air from the cabinet. Units within the rackcannot be upgraded without shutting down the entire unit.

The Stulz ‘CT Cooling Rack’ is a cooling system for electronicenclosures that can be fitted onto existing cabinets, and is mainlydirected to the PABX market in telecommunications. The cooling system isalso available with a cabinet comprising three sides and a glass door,with the cooling unit situated on top of the cabinet. Air inside thecabinet is cooled by ambient air that is drawn through a heat exchangerin’ the cooling unit and is then circulated within the cabinet. Again,an internal rack-mounted UPS is an option.

Neither of the Liebert or Stulz products is able to achieve the degreeof cooling required by a fully-filled 42U-plus cabinet in a large datacenter. Also, while their localized cooling provisions go some waytoward reducing the contamination and inefficiency issues of whole-roomcooling, they still have inefficient and ill30 defined airflow withinthe cabinet. For example, the top-down flow of cold air from top-mountedcooling units goes against the natural upward flow of warm air, andrisks condensation problems as moisture in the rising warm air meets thecold downward flow. Moreover, there is still a risk that some serverswill receive less cooling air than they should, and that failure of aserver's internal fan will result in overheating.

Returning to the power supply issue mentioned briefly, above, redundantengineering is often built into data centers when they are firstestablished so as to allow for future expansion. This inevitably resultsin waste of money and resources if the data center does not quicklyreach its expected capacity. Conversely, if the data center quicklyexceeds its expected ‘capacity, there is a lengthy lead-time becauseadditional power allocation after initial request from a data centertends to be extremely slow. This is a limiting factor upon the naturalgrowth of the data center.

The result is that tenants usually request more power than they needinitially, leading to a scalability problem because power infrastructureneeds to be installed for the entire data center from the outset dayeven if there are only a few tenants at that stage. It would bepreferable if there was increased flexibility of adding power to a datacenter at short notice so that tenants would only need to request extrapower as and when required.

Thus, correct sizing of appropriate technology is extremely important.If too much site infrastructure capacity is installed, then those makingthe investment recommendations will be criticized for the resulting lowsite-equipment utilization and poor efficiency. However if too littlecapacity is installed, a company's IT strategy may be constrained, ornew services may have to be outsourced because there is no space withsufficient site infrastructure capacity to do the work internally.

Summarizing all of the above, those skilled in the art know that thermalcharacteristics and airflow movement within a typical data roomenvironment are extremely complicated and, to a large extent, full ofuncertainty. As cooling loads increase, this becomes more critical.Conventional cooling solutions can cope up to, say, 2 kW to 3 kW percabinet, provided that cabling and other requirements are modest. Abovethis level, it becomes necessary to either spread equipment out widely,which may not be practical or cost effective, or to place restrictivelimits on the number of hot devices that can be deployed within a rack.It will be recalled in this respect that a typical maximum deploymentdensity is just 40% of rack space. Currently, such limits are oftenforced on users due to the action of thermal triggers within theelectronic equipment.

It is against this background that the disclosure has been presented.

SUMMARY

In one aspect, the disclosure provides a cabinet for housing a verticalarray of heat-producing units, the cabinet having an equipment chamberadapted to support the array such that the array cooperates with thecabinet in use to define a first plenum, the first plenum having aninlet for receiving a flow of cooling fluid and an outlet defined by aplurality of openings through the array whereby the first plenumcommunicates with the openings in use to exhaust substantially all ofthe flow of cooling fluid through the openings and hence through thearray, wherein the inlet to the first plenum admits fluid to the firstplenum in a substantially horizontal direction.

In another aspect, fluid flows across the first plenum in use as ahorizontally-moving curtain of fluid that is preferably substantiallyuniform from top to bottom across the array. This helps to ensure evenapportionment of cooling air between all of the heat-producing unitssuch as servers.

The inlet to the first plenum may be a substantially vertical slotbeside the first plenum, which preferably extends substantially the fullvertical extent of the array or of the plenum.

Advantageously, fluid passing through the array is recirculated forintake to the first plenum. To that end, a second plenum may be providedfor receiving the flow of fluid once that flow has passed through thearray, the second plenum having an inlet defined by a second pluralityof openings through the array, and an outlet.

The outlet from the second plenum may lead the fluid to a plant forcooling and impelling the fluid. That plant preferably includes at leastone heat exchanger and at least one impeller. It is possible for theheat exchanger to be either upstream or downstream of the impeller. Theplant may further include one or more filters for filtering the fluidbefore it returns to the first plenum.

The plant may include a single heat exchanger, which is relativelyreliable and has less need of redundancy, and a plurality of impellers,which are relatively unreliable and have more need of redundancy. Eachimpeller may be associated with a non-return valve that closes in theevent of failure of that impeller, preventing short-circuiting ofairflow through the failed impeller.

For compactness while efficiently promoting the aforesaid curtain ofair, the impellers are preferably disposed in a substantially verticalarray.

For ease of maintenance, especially where there is no redundancy ofequipment, it is preferred that at least the heat exchanger is a modulereplaceable during use of the cabinet. For example, the heat exchangermay be mounted to the cabinet on runners supporting the heat exchangerwhen it is withdrawn from the cabinet, and may be coupled to coolantsupply ducts by dry-break connectors.

In preferred, compact arrangements, the plant is housed in a plantchamber beside the equipment chamber. Fluid can circulate in use betweenthe plant chamber and the equipment chamber: for example, the flow offluid through the equipment chamber may be substantially parallel to andopposed to the flow of fluid through the plant chamber. It isadvantageous for the general flow of that fluid to be substantiallyhorizontal throughout said circulation. Nevertheless, it is preferredthat the general flow of fluid emanating from the plant chamberundergoes a substantially orthogonal direction change to enter the firstplenum and that the general flow of fluid emanating from the secondplenum undergoes a substantially orthogonal direction change to enterthe plant chamber.

At least one door suitably affords access to the plant chamberindependently of access to the equipment chamber. Thus, for example,respective doors affording access to the plant chamber and the equipmentchamber may have independent locks capable of permitting access to onechamber but not both, so that only suitably authorized personnel areallowed access to each chamber.

In preferred embodiments, each plenum extends substantially verticallybetween an upright wall of the cabinet and an upright face of the array,and openings through the array are distributed across the face of thearray. The openings through the array preferably extend substantiallyhorizontally between first and second plenums opposed about the array.Elegantly, the upright wall may be a door or removable panel affordingaccess to the cabinet.

The cabinet, in one embodiment of the disclosure, is preferably adaptedto house units in the form of servers. It may also house or be adaptedto house power supply and/or fire suppressant units, and may furtherinclude heat transfer means for carrying heat away from the cabinet.

In another aspect, the disclosure provides a method of cooling an arrayof heat-producing units housed 10 in a cabinet, comprising directing aflow of cooling fluid into a plenum that communicates with openings inthe array and confining the flow such that substantially all of the flowpasses from the plenum through the openings, wherein the fluid entersthe plenum substantially horizontally.

The method may comprise directing the flow across the plenum and/orapportioning the flow substantially equally among the openings. The flowinto the openings may be transverse to the direction of flow through theplenum, although the direction of flow through the plenum and the flowthrough the openings is still preferably generally horizontal.

As before, fluid preferably flows across the first plenum as ahorizontally-moving curtain of fluid such that the flow of fluid issubstantially uniform from top to bottom across the array, and fluidadvantageously recirculates in use within the cabinet with its generalflow direction being substantially horizontal throughout saidrecirculation.

In another aspect, the disclosure provides a data center installationcomprising at least one cabinet corresponding to an embodiment providedby the disclosure or operating according to an embodiment of thedisclosure. The installation may further include door interlock meanspreventing access to a cabinet if specified conditions are not met. Onesuch condition is user authority to access the cabinet. Another isenvironmental compatibility inside and outside the cabinet, to preventcondensation. Another is that an outer enclosure around the cabinet mustbe closed.

An outer enclosure around the cabinet preferably includes airconditioning means for controlling temperature and/or humidity aroundthe cabinet. That enclosure may be equipped with external panels spacedfrom walls of the enclosure to shade, insulate and cool the walls.

The cabinet to be described herein achieves energy savings in comparisonwith conventional equipment rack cabinets, where these are cooled usingpressurized raised floor systems in conjunction with close controlcomputer room air handling units (CRACs or CRAHs). The cabinet systempermits racks with very different heat/cooling loads to be locatedadjacent to each other without one affecting the other.

The heat loads generated by electronic devices often vary due to theoperational state of the device at any given time. The cabinet systemdescribed herein reacts to this by delivering only the amount of coolingthat is required at any given moment to the specific equipment in eachrack. The system is capable of utilizing all of the rack space for hotdevices as opposed to conventional rack/cabinets which are typicallylimited to 40% of the rack space. The system is capable of dealing withhigh-density loads of up to 15 kW per rack IT cooling load and beyond.

To maintain a closed environment, the cabinet is preferably sealed to anIngress Protection rating of ‘IP55’. These digits represent resistanceto water ingress from light jets of water played onto the cabinet, andresistance to physical ingress from airborne dust. Sealing the cabinetin this way prevents dust ingress which is often a problem in datacenters due to ongoing building or maintenance work taking place afterthe data center begins operation. Sealing also resists water ingress,especially in multi-tenant facilities where some tenants or techniciansmight carry out activities which result in water spills to those below.In particular, equipment cannot be damaged by water spraying or drippingthrough conventional perforated doors.

In some embodiments, the cabinet contains all the requirements for adata center environment (environmental/cooling control, fireprotection/detection, power management and security) within the cabinet.In its standalone variant, it provides a data center environment for useanywhere, internally or externally. Combined with its modularplug-and-play remote plant, the cabinet of these embodiments provides atruly scaleable solution that can be scaled both upwards and downwards.

The disclosure therefore enables the provision of a physically andenvironmentally safe and secure environment for locating existing andfuture IT/electrical critical technology in high-density deployment.High-density deployment in this sense means the ability fully to occupythe rackable area of the cabinet element 100% with hot devices forcooling loads between 3 kW and up to 20 kW per cabinet. Embodiments ofthe disclosure facilitate this deployment without the user having to beconcerned about the relative positions of the electrical equipmentwithin the rack space in regard to thermal performance. In particular,the stack or chimney effect is essentially eliminated due to thehorizontal airflow delivery to the equipment.

Due to the close proximity of the water-cooled heat exchanger to theheat source combined with the relatively wide ‘air duct’, rememberingthat conventionally-directed forced air systems flowing vertically havea relatively narrow duct, very little heat is transferred by convectionto the inner walls of the cabinet. Further, due to the preferred dualskin insulated core construction of the cabinet walls, any heat that istransferred is not conducted across the core to the outer skin.Similarly, heat gain or loss is not experienced from the room side ofthe cabinet to disrupt the precisely controlled environment.

Aspects of the disclosure address potential ‘cold bridging’ problems andthe quality of sealing: the majority of standard industry containmentsleak badly, sometimes even those claiming to be sealed. The cabinet willsatisfy the 1P55 rating as a minimum (higher ingress protection ratingscan be achieved if necessary) and can be considered ‘room neutral’. Thisis particularly important where a user wishes to locate a high-densitydeployment within an existing environment without causing problems tothe existing equipment. This secure environment is provided in a formthat permits its use in any location where it might reasonably berequired, that is, not only within a data room environment but also inordinary internal and external spaces such as offices, factories,warehouses, marine applications and on open sites such as may benecessary for military or construction use.

The containment of embodiments of the disclosure will safely house anyEIA-310-D 19′ rack mounted product with front to back ventilationincluding blade servers, from any vendor or OEM. The system architectureis designed to provide a minimum of n+1 redundancy for the servicehoused and high availability and fault tolerance. For example,components more likely to fail under risk analysis are designed to be‘hot pluggable’.

Automated control systems minimize the requirement for humanintervention which depending on the research considered can account forup to 40% of downtime. The control systems are designed to provideproactive remote monitoring and control to minimize the opportunity forunforeseen failure.

The system architecture provides true scalability both upwards anddownwards, allowing users to match current real needs with appropriatelevels of secure provision rather than over-engineering on the basis ofwhat will be needed or might be needed at some future date.

Attempting to achieve energy efficiency with a room-level systempresents a number of difficulties. For example, the delivery path forcooled air for example in a pressurized floor solution is subject tonumerous variables, such as under-floor obstructions, room surfacetopography and widely varying equipment loads across the room. A lengthyair path from plant to containment and return to plant, depending uponthe particular system used, risks inefficient use of the central plant,e.g., chilled air bypass, loss of static pressure and too high a staticpressure. By shortening the distance from the source of heat generationto the point at which heat is transferred via convection to a densermedium than air, and limiting the volume of the technical space to becooled (down to cabinet level) it is possible to deal efficiently withvery high heat loads.

Embodiments of the disclosure provide a pre-configured system where allthe interface problems between the elements have been dealt with andincorporated into a single system prior to delivery to the end user. Theprovision of certainty rather than the uncertainty of traditionalsystems allows users to meet their audit needs with ease under corporategovernance requirements.

BRIEF DESCRIPTION OF THE DRAWINGS

Reference has already been made to FIG. 1 of the accompanying drawings,which is a schematic sectional side view of a conventional data centerhaving equipment cabinets which are cooled by a close-control room airconditioning unit (CRAC unit) within the technical space.

In order that the disclosure be more readily understood, reference willnow be made, by way of example, to the remaining drawings in which:

FIG. 2( a) is a sectional plan view an example of a cabinet in anon-controlled environment, thus equipped with a secondary enclosure;

FIG. 2( b) is a sectional view of the cabinet and enclosure taken online Y-Y of FIG. 2( a);

FIG. 2( c) is a sectional view of the cabinet and enclosure taken online X-X of FIG. 2( a);

FIGS. 3( a), 3(b) and 3(c) correspond to FIGS. 2( a), 2(b) and 2(c) butshowing external environment panels outside the secondary enclosure;

FIG. 4 is a schematic diagram showing the power and fluid connections ofa system incorporating cabinets;

FIG. 5( a) is a sectional plan view of an example of a cabinet;

FIG. 5( b) is a sectional side view of the cabinet of FIG. 5( a);

FIG. 5( c) is a sectional rear view of the cabinet of FIGS. 5( a) and5(b);

FIG. 5( d) corresponds to FIG. 1 but shows the cabinet of FIGS. 5( a) to30 5(c) housed within the technical space and replacing one of thecabinets of FIG. 1;

FIG. 6( a) is a sectional plan view of an alternative cabinet;

FIG. 6( b) is a sectional side view of the cabinet of FIG. 6( a); and

FIG. 7 is a perspective view of a cartridge and shows how acartridge-type coil may be hot-swapped during maintenance or repair.

DETAILED DESCRIPTION

Referring firstly to FIGS. 2( a), 2(b) and 2(c), FIGS. 3( a), 3(b) and3(c) and FIG. 4, preferred embodiments of the invention comprise threemain elements which together enable an autonomous cabinet system 28, inthe sense of a data center facility within a single equipment cabinetthat does not rely on external service provision other than electricalpower and coolant connections. If required, power and coolant facilitiescan be provided using plant skids which may, for example, include agenerator set for the provision of electrical power. Suitably securemains power connections are of course possible and, in most cases,preferred.

The first element of the system is the equipment cabinet 30 itself,which is sealed from its immediate environment. When used innon-controlled environments 32 as in FIGS. 2( a), 2(b) and 2(c), thecabinet is placed within a secondary outer enclosure 34 which insulatesit from the environment and provides a zone where humidity can becontrolled by a small package cooling unit 36. An equipment cooling unit(ECU) 38 within the cabinet 30 provides cooling/heating and humiditycontrol and connects by a pipework system 40 to a plant skid 42 (shownin FIG. 4). The door 44 of the outer enclosure 34 is interlocked withthe cabinet door via the control system to prevent both being open atthe same time.

Further, if in an external environment, extra panels 46 may be addedoutside and spaced from the secondary enclosure 34 as shown in FIGS. 3(a), 3(b) and 3(c). These provide for passive cooling by virtue ofairflow through the gaps between the panels 46 and the enclosure 34 andalso ensure that the walls of the outer enclosure 34 are in shadeconditions. This reduces the cooling required for the outer enclosurespace 32.

Specifically, the extra panels 46 reflect direct solar gain and byvirtue of the gaps, they also provide a means of passiveventilation/cooling. In the event of solar gain experienced by thepanels 46, air heated in the gap between the panels 46 and the outerenclosure 34 creates a chimney/stack effect in which air enters belowthe bottom edge of the panel 46 and exits at the top edge. Thus, thereis a continuous supply of cooling fresh air and exhaust of warm airbefore that warm air can transfer significant heat to the outerenclosure. Similarly a horizontal panel may provide a through-ventedroof cavity. This ensures that the main insulated outer enclosurestructure remains in shade conditions reducing the amount of coolingrequired to maintain a suitable ambient environment.

Referring now to FIG. 4 in particular, the second element of the system28 is the pipework system 40 connecting the cabinet ECU 38 to the remotechiller plant skid 42. This comprises a prefabricated, insulatedpipe-in-pipe system to provide maximum protection against leakage. Thesystem 42 can be connected at high or low level to the cabinet 30 via aflexible pipe-in-pipe hose 48. The flexible hose 48 is connected to avalve box 50 which contains flow and return isolating valves (forindividual cabinet systems) or a prefabricated commissioning balancingset (for multiple cabinets). Rigid pipe-in-pipe 52 runs from the valvebox 50 to the chiller plant skid 42 itself.

The third element of the system 28 is the remote chiller plant skid 42comprising one of a range of modular skids sized to suit whateverpermutations of cabinet numbers are required. Though not shown, eachskid 42 is provided with two chillers (providing n+1 redundancy), abuffer tank, a mixing manifold, variable speed pump sets, an actuatorand a control panel.

This combination of elements allows for truly scaleable deployment,firstly cabinet-by-cabinet, and secondly by modular remote plant. Ascabinets are added in small increments over time, a point will bereached where the multiplicity of remote plant modules will not be sizedcorrectly in relation to the total load to provide maximum efficiency inrunning and maintenance costs. In the invention, the plant skids 42which are therefore designed on a ‘plug and play’ basis can be addedinto or withdrawn from the pipework system 40 without closing down theservice. This allows plant skids 42 to be swapped out at any time in thefuture, and more appropriate size modules added to maintain maximumefficiency with regard to running and maintenance costs.

In contrast with current data center technology, plants are typicallysized for the ultimate total load, which means that the plant may beoversized for periods of sometimes years until the actual loadapproaches that level. If, conversely, the plant is undersized for theeventual total load, then this may cause disruption to live servicesrequiring upgrading.

In terms of service connections, the cabinet 30 of the invention iscarried by the raised floor of the data room or directly by the solidfloor of the building as required. Electrical power cables are connectedto the cabinet 30 via panel mounted ‘commando’ plugs located both on thebottom and top of the cabinet 30 to allow either connection from theraised floor void or the ceiling void or other overhead services if asuspended ceiling is not fitted. There are four electrical connectionsto the cabinet:

-   -   (i) 32 Amp A and B ‘clean’ secured supply to power the dual        corded equipment rack Power Distribution Unit's (PDU's);    -   (ii) 16 Amp C1 and C2 ‘dirty’ secured supply to power the ECU        38. These supplies (clean and dirty) are separated to avoid any        possible earth noise problems being transmitted from the ECU 38        to the equipment rack PDU's. The C1 and C2 supplies are run via        the rack mounted FPU which contains a 16 Amp circuit breaker        which is opened in the event of a fire alarm condition to shut        down the cabinet fans. An emergency power off (EPO) link is also        run from the FPU to the PDU link boxes to shut down the PDU's in        the event of a fire condition. If a transfer switch or rackable        UPS is fitted this is also connected to the FPU's EPO.    -   (iii) Between the A and B electrical inlet plugs and the        equipment PDU's two link boxes (A and B) are located in the        bottom of the racked space which constantly monitor the RMS        voltage, RMS current, and kWh of each PDU. Each of the        individual socket outlets (IEC 1OA as standard one socket outlet        for each U position of rack space) is remotely switchable (no        switches are fitted to the PDU strip to avoid personnel        accidentally switching off the wrong service).]    -   (iv) The link boxes are provided with an LCD display of the        instantaneous RMS current, RMS voltage, and cumulative kWh. A        32A Class C Double Pole MCB provides over current protection. A        communication port (EIA RS485) is 5 provided for individual        socket switching, data input/output and power and a data        programming port (RJ45).

There are two chilled water connections onto the cabinet made with ‘drybreak’ connectors and flexible hoses (pipe-in-pipe) 48, either at lowlevel or at high level. The flexible hoses connect to the sealed valvebox 50, with the outer hose screwed onto the housing of the box whilethe inner hose passes through the box to connect to valves therein. Thevalve box 50 contains either isolation valves (single cabinet) or abalancing/commissioning set (multiple cabinets). Thus those items whichmight possibly give rise to a leak, namely the valve connections/valvebodies, are contained within a leak-proof enclosure.

A rigid pipe-in-pipe system 52 runs from the other side of the valve box50 to the plant skid 42. The pipe system 40 is supplied inpre-fabricated format (3 m/5 m lengths) comprising an inner triplelayered plastic/metal/plastic pipe to which insulation is bonded. Theouter corrugated pipe facilitates pulling back the outer sleeve andinsulation to make the pipe joint (either fusion welded or crimped). Thejoint, is made and a vapor seal collar applied over the joint.Adjustable pre-assembled pipe supports allow fixing of the outer piperun to the building fabric. Leak detection tape can be provided in theouter pipe linked to the cabinet controller. A buffer tank requiresfilling with water; but once filled does not require, a permanent coldwater feed.

Where it is required (and if there is sufficient cooling capacity) toconnect the cabinet system 28 into an existing facility chilled watersupply, then it is necessary to provide a plant module to raise thechilled water supply temperature to 11.5° C. before entering the cabinetchilled water main.

The cabinet system 28 can operate as completely stand alone, or have adata connection run from the cabinet controller to a local desktop PC, alocal control room, or via a LON Gateway or SNMP to communicate via anIntranet or Internet link for remote access. Similarly, the remote plantskid control panel can be linked to a local desktop PC, a local controlroom, or via a LON Gateway or SNMP to communicate via an Intranet orInternet link for remote access. The mains pipe-in-pipe leak detection(if fitted, being optional) is connected to cabinet controller.

Referring now to FIGS. 5( a), 5(b), 5(c) and 5(d), the cabinet 30 of theinvention is generally cuboidal and is constructed largely ofrectangular steel panels which may be structural, although the cabinet30 may also have an underlying structural frame to which the panels areattached. The panels define parallel horizontal top and bottom walls 56,58 and parallel vertical side walls 60, 62, 64, 66 extending between thetop and bottom walls 56, 58. A vertical partition 68 extends parallel tothe side walls, also between the top and bottom walls 56, 58.

The cabinet 30 comprises two main parts divided by the partition 68,namely a rackable equipment space 70 beside an equipment cooling unit orECU 38 (FIG. 2( a)). In other words, the partition 68 within the cabinet30 defines a server chamber 83 for racking beside a plant chamber 85 forimpelling and cooling air to pass through and between servers in theracking.

The partition 68 does not extend to the full depth of the side walls 60,62, 64, 66 or the top or bottom walls 56, 58, therefore leaving gaps orslots at the front and rear of the partition. These gaps or slotsprovide for recirculating airflow between the server space and the plantspace, via a front supply plenum extending over the front face of a bankof servers supported by the racking and a rear exhaust plenum extendingover the rear face of that bank of servers.

The invention therefore contemplates a cabinet carcass forming twointernal areas; an equipment rack space 70 and an ECU space.Advantageously, the part of the carcass defining the ECU space isremovable from the part of the carcass defining the rack space 70 toallow for easier installation access in existing buildings with limiteddoor opening width. However, this is not essential to the invention inits broad sense.

The carcass has a double-skin construction to reduce weight, providestructural integrity, reduce noise transmission, reduce thermaltransmission and increase security. The carcass may be constructed fromany of a variety of materials to best suit specific applications, or acombination of them, for example steel, aluminum or plastic skins withmineral wool filling, aluminum honeycomb, high density foam or synthetichoneycomb cores.

The cabinet 30 contains racking 72 defining bays capable ofaccommodating a corresponding number of 1U units such as servers. Ofcourse, deeper units of 2U or more in thickness can be accommodated ifthe overall number of units in the cabinet is decreased. The units arepositioned close together in a layered stack-like configuration,although the units are supported from the sides of the cabinet and arenot actually stacked in the sense of resting upon one another. Thismeans that units can be removed and replaced without disturbing adjacentunits above or below.

Some capacity in bays at the bottom of the cabinet 30 may be devoted toan electrical power management unit such as a UPS and a further capacityin bays at the top of the cabinet may be devoted to a gas firesuppressant unit. This leaves the remaining capacity for other unitssuch as servers protected by the power management unit and the firesuppressant unit.

The fire suppressant unit may, for example, be a gas dump unitcontaining heptafluoropropane suppressant, as is commonly sold under thetrade mark FM200 of Great Lakes Chemical Corporation and knowngenerically as HFC-227ea. Gas dumping can be triggered by a smokedetector such as is sold under the trade mark VESDA of Vision Systems'Group.

In conventional manner, each server within the cabinet 30 defines anairflow path between ventilation openings such as grilles in its frontand rear faces, which openings may be referred to as front ventilationopenings and rear ventilation openings respectively. There may of coursebe other openings in the top, bottom or sides of servers. Each servertypically also includes an impeller to promote cooling airflow alongthat path around heat sources within the server.

It will be apparent that each cabinet 30 defines a sealed environmentthat, in emergency situations, has an important element ofself-sufficiency in terms of cooling, fire protection and power supply.To that extent, each cabinet 30 is a mini data center that is apt to beretro-fitted to an existing site, and that can be filled to its maximumcapacity without overheating as will be discussed in more detail later.

Access to the interior of the cabinet 30 is via four doors, two on thefront 74, 76 and two on the rear 78, 80. One door of each pair 76, 80gives access to the ECU 38 and the other door of each pair 74, 78 givesaccess to the equipment rack. The doors are side-hinged and sealedaround their periphery. They may be glazed although that is nottechnically significant.

The doors 74,78 giving access to the equipment rack 70 are spaced fromthe front and rear of the server units so that in conjunction with theside panels 60, 66 and the partition 68, they create a front supplyplenum 82 communicating with the front ventilation openings of theserver units and a rear exhaust plenum 84 communicating with the rearventilation openings of the server units.

The rear exhaust plenum 84 is closed to all sides but one, where itcommunicates with the plant chamber 85 through a gap or slot 86 at therear of the partition, thereby to exhaust air that has been warmed byits passage through the server units. That air is cooled, filtered andimpelled through the plant chamber 85 into the front supply plenum 82via a gap or slot 88 at the front of the partition. Like the rearexhaust plenum 84, the front supply plenum 82 is closed on all othersides.

Substantially all of the incoming air must pass through the front plenum82 and from there through the front ventilation openings of the servers.To ensure this where the cabinet is not full, blanking plates should befixed across any bays not occupied by servers; otherwise, air would flowpreferentially through the resulting gaps, around rather than throughthe servers.

Each door 74, 76, 78, 80 is lockable by electric (preferably magnetic)door locks under smart card control, to which end a smart card reader(not shown) is provided on the front and rear of the cabinet. Smartcards may be programmed to give access to either the ECU doors, theequipment rack doors 74, 78 or all doors depending on the duties of thepersonnel issued the card. Additionally, cards can be programmed tooperate the access doors to the room where the cabinet 30 is located andall the other access doors en route to it.

At the base of the cabinet 30, a secure drawer unit 90 housesprogrammable control systems that operate the system 28. However, thispositioning is not essential: other variants or models may locate thecontrol system elsewhere, for example within the ECU space or doormounted. Whatever the position, the principle is the same in that accessto the controls is possible without having to enter the rackable space70.

Where different personnel maintain the ECU/controls and the rackablespace, neither must have free access to the other's area ofresponsibility to avoid operational/maintenance incidents which mightresult in downtime. Consequently, upon presentation of the smart card tothe smart card reader, the programmable controller inside the cabinet 30checks with a security record that the user is authorized to enter thecabinet.

Having confirmed this, the controller then uses sensors to check theexternal and internal environment, which should be similar although thecabinet environment is more precisely controlled. If there is adiscrepancy between the external and internal environment that couldresult in a dew point problem when the doors are opened, then theelectric door locks are not released. Otherwise, room air can depositmoisture either within the cabinet 30 to be carried by airflow onto theracked equipment or directly onto the racked equipment itself. So, inthe event of such discrepancy, a warning is given by a light or buzzerto the person trying to gain access, alerting them that the cabinetenvironment must be adjusted first.

To harmonize the external and internal environments, the controller mayuse variable speed fans and chilled water valves to adjust the internalenvironment to eliminate the problem. Once this has been achieved, thewarning indication ceases and the door locks release. Should theexternal and internal environments be too far apart to harmonize in thismanner, then the door locks will not release. The user then has toaddress the reason for the external data room environment having movedso far outside its specified limits. Should the reason for denyingaccess be due to a fault with the controller, this can be verified viaanother alarm (general) condition. If this is the case, then it ispossible to open the doors with a manual override key which should beheld at a separate security point. Manual locks or latches may beprovided in addition to the electric locks to ensure that door seals aremaintained at all times when the doors are supposed to be closed.

Once the purpose of entry to the cabinet 30 has been completed, the usercloses the 10 door(s) and re-presents the smart card which locks thecabinet 30 and puts the ECU 38 into ‘soft start’ mode. Soft start isused at initial commissioning to bring the internal environment back upto set point over a timed delay (normally 15 to 20 minutes) to avoid anydew point problems with the room air that has been introduced into thecabinet.

The ECU space contains cooling equipment comprising a chilled water coil92 (heat exchanger) and a vertical array of fans. The relationshipbetween the fans 94 and the coil 92 varies between variants. In thefirst variant shown in FIGS. 5( a) to 5(c), the fans 94 are positionedat the rear of the cabinet 30 and draw air from an exhaust plenum 84 atthe rear of the rack space 70. The airflow is then pushed into a middleplenum 96 and then through the coil 92 and filters 98, downstream of thefans 94, to the front of the cabinet 30, where it flows into the supplyplenum 82. The supply plenum 82 delivers the airflow to the front of theracked equipment, where it passes through the equipment ventilationholes, collecting heat from the electronic components and exhaustinginto the exhaust plenum 84 to start the cycle again.

It will be appreciated that the air flow circulates continuously in ahorizontal pattern akin to the movement of a curtain. This movementpattern avoids problems with stack/chimney effect, as each device isdirectly fed with cooled air from the coil 92.

This means that unlike all vertical airflow systems, it is no longercritical where the hottest devices are placed. The horizontal airflowalso encounters less problems with cabling resistance, which is anincreasing problem for containment due to the effects of compactionalready noted.

Moreover, the invention provides a much greater ‘duct area’ than ispossible with a vertical system. Consider that the effective duct widthfor a conventional vertical system is set by the overall width of thecabinet enclosure (600 mm) although normally, due to structuralrequirements, the actual width is inside the rack rails, namely 500 mmor less. Ignoring systems which place the duct opening directly underthe rack, the duct height depends upon the space available in front ofthe rack. This space can be as little as 30 mm in some cases; whereasfrom tests carried out with a variety of airflow areas, the minimum ductheight should be 75 mm to 100 mm.

Even assuming a duct height of 100 mm is provided across the full 600 mmwidth of the cabinet enclosure, then the maximum effective duct area forconventional vertical airflow is just 0.1×0.6=0.06 m2. In comparison,the horizontal airflow of the invention enables full use of the cabinetheight as the effective duct width. For example a 42U version of thecabinet has a duct extending for 1.9 m in cabinet height. Thus, for thesame duct height of 100 mm, the effective duct area is 0.1×1.9=0.19 m²or over three times that of the conventional vertical system.

The cabinet 30 of the invention also benefits from markedly lowerresistance to airflow. The horizontal airflow system of the inventionrequires four changes of direction to complete a full cycle whereasvertical airflow employing a central plant requires ten, made throughmore restricted ducts. Thus, the horizontal airflow system makes itpossible to provide greater airflow to deal with very high loads; withless system resistance to airflow.

In the invention, the proximity of the cooling unit 38 to the equipmentbeing cooled means that very little heat is transferred from the airflowto the inner walls of the cabinet. This, combined with the sealedenvironment, ensures that the cooling loads generated by the housedequipment do not influence other equipment close by. The more limitedenvironmental area allows more precise automatic cooling to the levelnecessary at any given moment thus minimizing power consumption, andremoving reliance on human intervention that is required with manyexisting cabinet enclosures.

In a second variant of the invention as illustrated in FIGS. 6( a) and6(b), the fans are positioned at the front of the cabinet 30 b and pullair through the coil 92 which is located upstream of the fans, towardsthe rear of the cabinet 30 b. The airflow is then as above, moving intothe supply plenum 82; through the racked equipment; into the exhaustplenum 84 and then back through the coil 92 to start the cycle again.

In the preferred embodiments illustrated in FIGS. 5( a) to 5(c) andFIGS. 6( a) and 6(b), six fans 94 are arranged in a vertical array topush or pull the airflow through the coil 92. Five fans 94 are neededfor load with one for redundancy in an n+1 arrangement. The number offans 94 is directly related to the total cooling load and coilconfiguration. Currently a total of six are used for models which havetotal capacities of 15 kW to 20 kW of IT cooling load. Lower loads mayrequire less fans but the principle is the same.

It is desirable that all fans 94 should run all the time, because fansare more likely to fail on start-up, especially if they have not beenturned over regularly during maintenance. Should a fan fail, anon-return flap 100 closes over the failed fan to prevent ‘shortcircuiting’ of the airflow, whereupon the remaining fans speed up totake up the load. This non-return valve 100 feature is advantageous inthe first embodiment where the fans are upstream of the coil, but is notnecessary in the second embodiment where the fans are downstream of thecoil.

The fans 94 are hot-swappable requiring the release of quick-releasefittings and an electrical plug connector in a process that involvesapproximately four minutes to swap out a fan.

Monitoring equipment can detect increased power consumption by any fan,94 indicating a possible future fan failure and allowing the unit to beswapped before the failure occurs. The combination of variable speed andchilled water valves linked to sensors permits efficient cooling levelsto be maintained. In other words, only the level of cooling required ofthe mechanical equipment is delivered automatically at anytime.

N+1 redundancy is important for the fans 94, which are the most likelycomponents to fail but is less important for the coil 92 which rarelyfails. In any event, providing two coils 92 to achieve the same (n+1)level of redundancy as the fans would increase air resistance throughthe system, requiring larger fans and increasing power consumption.

Coils rarely fail, but when they do it is sometimes catastrophically oncommissioning or more likely as a result of a blow hole. Brazing fluxlodged in a hole may not be revealed with a factory air test, but willthen fail when filled with water upon commissioning. These incidents arevery rare but not unknown, so the invention contemplates providing n+1redundancy on the service but not the coil itself This is achieved bydesigning the coil as a cartridge which can be hot-swapped withoutshutting down the cabinet. To this end, the coil 92 complete withsolenoid water isolation valves and two (or three) port modulatingchilled water valves is made as an assembly 102, as shown in FIG. 7. Theassembly 102 is mounted on telescopic rails 104 and connected to theflow and return pipework via ‘dry break’ connectors 106 in which aninner valve closes before an outer coupling releases to avoid anycoolant spillage.

Monitoring procedures detect leaks and pressure loss within the coil 92.In the event of a coil failure, an engineer opens the doors to the ECUspace leaving the fans 94 running. Room air continues to circulatethrough the racked equipment which might rise in temperature but willstay within its operational limits. By maintaining some airflow duringcoil swapping, the suddenness of temperature rise within the cabinet 30is minimized and hence the risk of thermal shock damage to the equipmentprotected by the cabinet 30 is reduced. A thermal shock ‘spike’representing a rate of temperature rise of 10° C. per hour is consideredacceptable in this context.

Once access is gained by opening the doors 76, 80 to the ECU space, thedry break 92 lifted off the rails and replaced. The procedure isreversed with a new coil, the coil bled and the doors closed to resumenormal operation. It is envisaged that the total time necessary to swapa coil will be less than about ten minutes.

The area in the ECU space below the coil cartridge 102 is tanked so thatin the event of a spillage the contents of the coil 92 and the cabinetpipework etc are contained. Leak detection sensors within the tankedarea provide an alarm condition in this situation. In the event of acatastrophic leak, the chilled water valves automatically close toprevent more fluid entering the cabinet enclosure (this facility can bedisabled if required). In standard format, the outer hose of theaforementioned pipe-in-pipe system can be used as a drain. However if afire-rated cabinet is required then this hole is fire-sealed and it isnecessary to drain the ‘tanked’ area manually.

As mentioned above in relation to the door lock system, the coolingsystem is designed to maintain the cabinet environment above dew pointto prevent condensation forming on the heat exchanger coil and beingcarried into the racked equipment by airflow or forming directly on thesurface of the racked equipment. Design set point for the heat exchangerwater inlet temperature is 11.5° C. with a 16.5° C. outlet temperature.The sealed environment of the cabinet means the external dew point canbe ignored other than when the cabinet doors are opened, when theinterlocking of the door locks and the environmental controls (describedabove) prevent dew point problems.

The cabinet ECU 38 in combination with the cabinet 30 provides a closedloop air/water system dealing with sensible heat only. For this reasonthere is no dehumidifier within the ECU 38. The continuously circulatingair is drawn originally, and from time to time during operational andmaintenance access from the room air. In a data center, this air will bemaintained within prescribed humidity levels—normally 50% relativehumidity (RH) plus or minus 5% from the central make up fresh airsystem. Some OEM specifications allow for a much wider humiditytolerance while quoting the figure of 50% RH as ideal. While too highhumidity is to be avoided to prevent problems with condensation onequipment, too low humidity levels are also undesirable to avoidpotential problems with static electricity.

During the last few years, there has been an increase in the number ofequipment component failures due to humidity problems. This stresses theneed to target the environment within an ideal humidity tolerance band.Where the cabinet 30 is to be located outside data center environments,i.e., lacking close temperature and humidity control, care must be takento ensure the ability of the control system to prevent condensation viadoor interlocking is still viable. In other words the internal andexternal environment must be capable of being matched to stay above dewpoint but also maintain adequate cooling conditions for the equipment.

Where there is any doubt as to this requirement, then an outer enclosureshould be used as illustrated in FIGS. 2( a) to 2(c) and FIGS. 3( a) to3(c). This provides an insulated outer zone which is provided with asmall package HVAC unit 36 to maintain a stable ambient environment of22° C. 50% RH. The unit 36 provides cooling, heating (if required) andhumidity control and is linked to the plant skid 42 by a similar butindependent pipework system.

If the interconnecting pipe-in-pipe flow and return mains linking thecabinet heat exchanger to the plant skid 42 are not insulated, then asensor is attached to the pipe.

Thus, in the event of the measured room dew point approaching the fluidtemperature set point, the skid control panel will by means of the skidactuator and variable speed pump raise the fluid temperature say 1° C.or more to avoid condensation forming. However in standard format thepipe-in-pipe system is supplied pre-tested and insulated, so thisfacility is not needed. The skid primary chilled water circuit is 7.5°C. on supply.

The cabinet 30 of the invention is provided with dual-corded A and Bpower supplies as shown in FIG. 4, power monitoring and controlfacilities, and dual-corded C1 and C2 utility power supplies to the ECU38.

The cabinet 30 of the invention may contain various internal featureswhich are not essential to the invention and are not shown. For example,each power distribution unit (PDU) within the cabinet may contain an IECsocket outlet (a range of other outlets is possible to suit the countryof location) which is numbered and has a status neon lamp. If required,8U high modules (8 socket outlets) can be provided with individualsocket power monitoring.

Another internal feature not shown is a rack-mounted fire protectionunit (FPU) which provides an in-cabinet microprocessor-controlledsub-system for extinguishing fires within the cabinet 30. Fire detectionis provided by an in-cabinet laser smoke detection unit. FM200extinguishing agent (in a dual bottle arrangement) is preferred as thisagent is electrically non-conductive and not harmful to electronicequipment or to personnel. In the event of a fire situation detected bythe smoke detection unit, only the individual cabinet 30 is flooded withextinguishing agent and shut down rather than the whole room. After afire, extinguishant gas and fire residue may be extracted from thecabinet using a mobile gas bottle and vacuum pump, connected to a tapoff valve on the cabinet side. This also removes the need to installhigh and low level extract ductwork, complete with dampers and fansrequired for room level solutions.

The invention minimizes the impact of fire on the user's service, andminimizes the cost: say a replacement cost of $340.00 for gas as opposedto perhaps $136,000.00 for flooding a whole room of area 1,000 m², letalone the cost of downtime and possible equipment damage involved inflooding the whole room. Indeed, a suitably sensitive early warningdetection system provides control personnel with the option to shut offthe power to the rack, which will normally prevent a potential fire,before flooding the cabinet 30 with extinguishing agent.

An increased risk of fire follows from the process of compaction,requiring users to 20 consider their fire strategy. The value ofbusiness interruption for many users is far greater than the capitalcost of equipment loss. The automatic system installed in the cabinet ofthe invention protects the racked equipment and limits the damage to onerack. Being a sealed cabinet, the risk of cold smoke damage to otherequipment/services in the room is removed. In contrast, the majority ofdata rooms use a form of total flooding (either gas or water mist) toprotect the room space directly but the rack interiors and equipmentindirectly.

The invention has further benefits. For example, the cabinetconstruction of the invention together with its security systemsprovides a very high level of physical security required by many usersand their insurers.

Moreover, by obviating raised floors, the invention avoids otherproblems such as the problem of metal whiskers, namely swarf from theedge of the floor tile cut-outs which may be carried by the airflowsystems into the racked equipment of unsealed floor-ventilated rackcabinets.

The continuing increase in equipment and cable weight has the effect ofincreasing the loading within rack cabinets and therefore onto theraised floor. The full load capability of a raised floor is onlyrealized when all of its tiles are in place. In other words, the lateralstrength of the floor depends upon the presence of the tiles. As‘discussed above, tiles are often missing in many data centers. Theincreased load on the raised floor increases the point loads on thestructural floor, often beyond acceptable limits.

For users in earthquake zones, raised floor systems create an additionalhazard. While all systems are liable to experience downtime of hours ordays due to loss of connectivity in the event of an earthquake,collapsed raised floors result in racked equipment damage which canextend downtime to more than a month.

Operation of the invention will now be described in more detail. Thefunction of the plant is to maintain air in the data cabinet supplyplenum at 22° C. 50% RH. The basic temperature ‘set point’ is 22° C.which can be adjusted via an optional remote display and adjust panel(not shown). All other parameters to tune the control loops can also beadjusted via the optional local display and adjust panel.

Due to temperature stratification in the supply air plenum 82, theaverage of two temperatures is used to ensure that the supply airtemperature is adjusted to counter the mean cooling load in the cabinet.The supply air temperature set point is adjusted down from 22° C. to 20°C. when the average return temperature exceeds 34° C. Should any of thenecessary sensors be unreliable, i.e., open or short circuit, it isremoved from the averaged calculation.

The average of the two supply temperatures is compared with a slidingset point produced by the return air average temperature exceeding 34°C. The chilled water valve will then be modulated in accordance with aproportional plus integral control algorithm to maintain the set point.

All digital inputs are normally open for the fail condition to ensurethat the wiring circuit integrity and circuit breakers are alsomonitored. The alarm output is switched off for an alarm condition, forthe same reason.

Each variable volume fan maintains a constant static pressure under thecontrol of a proportional plus integral control algorithm, using theduct static pressure transmitter as input. Should this transmitter beunreliable, the fan speed will be controlled at a fixed value. Followinga power failure, the fan speed will ramp up gradually.

Each of the variable volume fans runs continuously unless disabled byany of the following conditions, namely:

-   -   ‘Gas Gone’—if the in-cabinet fire protection gas dump system is        active; if smoke input from the in-cabinet fire detection system        is active;    -   if the local isolator/alarm reset switch is off (hardwired into        the controlled equipment); or    -   the fan's respective fault condition is detected (hardwired into        the controlled 20 equipment).

The chilled water valve will be forced 100% open if any of the followingconditions is active, namely the supply fan is disabled or if all of thereturn air temperatures are unreliable (either open or short circuit).

The solenoid water isolating valves will be switched. off if water isdetected within the unit.

In terms of security function, a ‘door open panel light’ will illuminatewhen all the 30 following conditions are active:

-   -   there is a request from the control room system (if connected)        and the cabinet card reader;    -   the dew point temperature in the cabinet is higher than the        dew-point required to condense moisture from air entering the        unit when the doors are opened; and    -   there is no signal from the ‘Gas Gone’ alarm of the fire        protection (extinguishing) unit.

The ‘door open light panel light’ will flash when all of the followingconditions are active:

-   -   there is a request from the control room system (if connected)        and the cabinet card reader;    -   the dew point temperature in the cabinet is being adjusted to        prevent condensation of moisture from the air entering the unit        when the doors are opened; and    -   there is no signal from the ‘Gas gone’ alarm of the fire        protection (extinguishing) unit.

The ECU door magnetic locks will open when all of the followingconditions are active:

-   -   there is a request from the control room system (if connected)        and the cabinet card reader;    -   the dew point temperature in the cabinet is higher than the dew        point required to condense moisture from the air entering the        unit when the doors are opened; and    -   there is no signal from the ‘Gas gone’ alarm of the fire        protection (extinguishing) unit.

The equipment rack space magnetic locks will open when all of thefollowing conditions are active:

-   -   there is a request from the control room system (if connected)        and the cabinet card reader;    -   the dew point temperature in the cabinet is higher than the dew        point required to condense moisture from the air entering the        unit when the doors are opened; and    -   there is no signal from the ‘Gas gone’ alarm of the fire        protection (extinguishing) unit.

Moving on now to the Fire Detection and Protection System, the fireprotection unit (FPU) has a lockable isolating switch, for use when workis being carried out in the cabinet. If the unit is locked off, thecommon fault will be activated. This common fault will not include thelow gas pressure alarm, which is a separate input. When the air samplingsmoke detection system gateway is included, the smoke input will comefrom a LON SNVT which will replace the hardwired connection.

In a manual condition, which assumes that a control room system isconnected, the remote manual gas dump will be enabled if the followingconditions are active:

-   -   the control room has authorized that this function is active via        a network connection;    -   the smoke input from the air sampling smoke detection system        unit is active;    -   the door magnetic locks are not released (ECU or equipment        rack);    -   the pre-alarm input from the air sampling smoke detection system        unit is active; and    -   the fans are off.

In an automatic condition, which assumes a stand-alone configuration inwhich a control room system is not connected, the remote manual gas dumpwill be enabled if the following conditions are active. This is also abackup system to the remote manual gas dump, if the control room has notauthorized the function within a given time, and the other inputs arestill active:

-   -   the first knock is active (hardwired input from the air sampling        smoke detection system to the FPU);    -   the pre-alarm input from the air sampling smoke detection system        unit is active;    -   the second knock (smoke input from the air sampling smoke        detection system 15 unit) is active;    -   the door magnetic locks are not released (ECU and equipment rack        sections);    -   the fans are off; and    -   a configurable time delay (0 to 120 sees) has elapsed.

Moving on now to alarms, a ‘common plant alarm’ will be enabled if anyof the following conditions is active:

-   -   there is a fire! smoke alarm from the air sampling smoke        detection system unit;    -   there is a fault signal from the air sampling smoke detection        unit;    -   there is a pre-alarm from the air sampling smoke detection unit;    -   the ‘Gas gone’ signal is active from the FPU;    -   there is a ‘low gas’ signal from the FPU;    -   there is a common fault signal from the FPU;    -   the filter is dirty;    -   the front door status does not match the commanded position        (following a grace period of 5 minutes—only if the control room        is connected);    -   the back door status does not match the commanded position        (following a grace period of five minutes—only if the control        room is connected);    -   water is detected within the unit;    -   vibration is detected within the unit;    -   the temperature ‘set-point’ is not being maintained (plus or        minus 2° C.) following 30 minutes after a power failure; or    -   the cabinet humidity is less than 45% RH or greater than 50% RH,        following minutes after a power failure.

The above output will latch until reset via a control room system (ifconnected) or from the portable display and adjust panel.

A “fire alarm’ (flashing lamp on’ the panel) will be enabled if there isa fire/smoke’ alarm from the air sampling smoke detection unit. Thisoutput will also latch until reset via a control room system (ifconnected) or from the portable display and adjust panel.

A ‘gas gone lamp’ will be enabled if there is a ‘gas gone’ signal fromthe FPU (this is a hardwired signal) and this will indicate which unitthe alarm relates to when several units are connected together.

It will be apparent to those skilled in the art that the invention hasvery numerous and considerable benefits over the prior art. It providesa safe and secure total environment for locating existing and newIT/electrical critical technology in high-density deployment. Thisenvironment is provided in a form that can be utilized in any locationwhere it might reasonably be required: it is not necessarily dependenton a conventional data room location. The environment is also providedin a form that permits full use (i.e., 100%) of the equipment space forhot devices if required.

The environment systems of the invention provide high availability andfault tolerance 10 both under operational and maintenance conditions.The environment is ‘room neutral’ i.e. the cabinet of the invention doesnot contribute to any additional cooling loads or receive any additionalcooling loads from its surrounding space. It provides means for remoteproactive monitoring and control of the environment systems to ensuremaximum uptime. It removes as far as possible the need for personnel toschedule the order of deployment (stacking) of equipment for goodthermal management. It automates as far as possible the environmentcontrol systems to avoid the need for manual intervention and theresulting risks of downtime. It provides true scalability upwards anddownwards across all environmental systems, while maintainingenvironmental conditions suitable for the correct operation of allvendor/OEM products. The invention allows efficient energy consumptionboth for operational and maintenance requirements throughout the wholelife of an installation and at any given stage of build-out.

The invention provides a seamless means of avoiding the ‘fuzzy edgedisease’ of the industry, in the words, the interface problems arisingbetween traditional complex systems provided from a variety ofspecialist sources, especially high costs, increased timescales, loweravailability, multiple points of failure and long mean times to repair.It provides certainty to users, removing as many of the traditionaluncertainties and variables as possible and thereby simplifying thedecision/design process when configuring a facility.

In general, the invention may be embodied in many forms. Whendetermining the scope of the invention, reference should therefore bemade to the appended claims and to other conceptual statements herein,rather than the foregoing' specific description.

1. A data center system including: a heat exchanger; a cooling mechanismincluding an impeller array comprising a plurality of verticallyarranged impellers such that the impellers are arranged vertically oneabove another in use; and a substantially sealed, substantially airtightcabinet sized for housing a vertical array of heat-producing units, thecabinet having an exterior shell and the system including an interiordivider wall disposed inside the cabinet, the shell and divider wallproviding a heat exchanger chamber in which the heat exchanger isdisposed, the shell and divider wall providing an equipment chamberseparate from the heat exchanger chamber and adapted to support thearray of heat-producing units such that the array of heat-producingunits cooperates with the shell and divider wall in use to define afirst plenum, the first plenum having a first inlet defined by thedivider wall for receiving a flow of cooling gas and having a firstoutlet defined by a plurality of openings through the array whereby thefirst plenum communicates with the openings in use to exhaustsubstantially all of the flow of cooling gas through the openings andhence through the array, wherein the divider wall is configured suchthat the first inlet at least partially vertically overlaps with theplurality of openings through the array to allow the first inlet toadmit the gas through the first plenum in a substantially horizontaldirection; wherein the plurality of vertically arranged impellers aredisposed in the heat exchanger chamber vertically overlapping with thefirst inlet, the plurality of vertically arranged impellers configuredto horizontally impel a substantially uniform curtain of gas directly toor directly from the heat exchanger and to the first inlet and impel thegas substantially horizontally during an entire circulation of the gasthrough the equipment chamber and the heat exchanger chamber; andwherein the impellers provide N+1 redundancy of gas impelling.
 2. Thesystem of claim 1 wherein the divider wall is configured such that thefirst inlet will admit the gas over a substantial vertical length of thecabinet.
 3. The system of claim 1 wherein the divider wall is configuredsuch that the first inlet will admit the gas substantially uniformlyover a vertical length of the first inlet.
 4. The system of claim 3wherein the first inlet is at least one substantially vertical slotbeside the first plenum.
 5. The system of claim 3 wherein the firstinlet extends substantially a full vertical extent of at least one ofthe array and the first plenum.
 6. The system of claim 1 wherein asecond plenum is defined between the shell and the array for receivingthe flow of gas that has passed through the array, the second plenumhaving a second inlet defined by a second plurality of openings throughthe array, and a second outlet defined by the divider wall such that thegas is directed horizontally from the equipment chamber.
 7. The systemof claim 6 wherein the cabinet shell and divider wall are configured todirect the gas that has passed through the array to the coolingmechanism.
 8. The system of claim 7 further comprising a plurality ofheat exchangers.
 9. The system of claim 1 further comprising at leastone filter configured to filter the flow of gas.
 10. The system of claim1 wherein the heat exchanger is upstream of the impeller array.
 11. Thesystem of claim 1 wherein the heat exchanger is downstream of theimpeller array.
 12. The system of claim 1 wherein each impeller isassociated with a non-return valve that closes in the event of failureof that impeller.
 13. The system of claim 9 wherein at least a firstheat exchanger of the plurality of heat exchangers is a modulereplaceable during use of the array of heat-producing units and system.14. The system of claim 13 wherein the first heat exchanger is mountedto the cabinet on runners configured to support the first heat exchangerwhen the first heat exchanger is withdrawn from the cabinet.
 15. Thesystem of claim 9 wherein at least a second heat exchanger of theplurality of heat exchangers is coupled to coolant supply ducts bydry-break connectors.
 16. The system of claim 1 wherein the flow of thegas through the equipment chamber is substantially parallel to andopposed to the flow of the gas through the mechanism chamber.
 17. Thesystem of claim 1 wherein the cabinet includes a mechanism-access doorconfigured to provide access to the mechanism chamber without providingaccess to the equipment chamber.
 18. The system of claim 17 wherein thecabinet further comprises an equipment-access door, wherein themechanism-access and equipment-access doors have independent locks andare each capable of permitting access to only one of the equipment andthe mechanism chambers, respectively.
 19. The system of claim 18 whereinthe doors provide substantially vertically upright walls of the cabinet.20. The system of claim 1 further including heat transfer means disposedin the cabinet for carrying heat away from the cabinet.
 21. The systemof claim 1 further comprising a door configured to provide selectiveaccess to the heat-producing units based on at least one of (1) anenvironmental compatibility inside and outside the cabinet, and (2)whether an outer enclosure around the cabinet is closed.
 22. The systemof claim 1 further comprising an outer enclosure disposed around asubstantial portion of the cabinet.
 23. The system of claim 22 furthercomprising an air conditioner disposed and configured to control atleast one of temperature and humidity of air between the cabinet and theouter enclosure.
 24. The system of claim 22 wherein the outer enclosureincludes external panels displaced from walls of the outer enclosure.25. The system of claim 1 wherein the impeller array is configured suchthat any of the impellers can be replaced while the other impellerscontinue to operate.
 26. The system of claim 25 wherein each of theimpellers is removably connected to the cabinet with quick-releasefittings.
 27. A data center system including: a heat exchanger; acooling mechanism including an impeller array comprising a plurality ofvertically arranged impellers such that the impellers are arrangedvertically one above another in use; and a substantially sealed,substantially airtight cabinet sized for housing a vertical array ofheat-producing units, the cabinet having an exterior shell and thesystem including an interior divider wall disposed inside the cabinet,the shell and divider wall providing a heat exchanger chamber in whichthe heat exchanger is disposed, the shell and divider wall providing anequipment chamber separate from the heat exchanger chamber and adaptedto support the array of heat-producing units such that the array ofheat-producing units cooperates with the shell and divider wall in useto define a first plenum, the first plenum having a first inlet definedby the divider wall for receiving a flow of cooling gas and having afirst outlet defined by a plurality of openings through the arraywhereby the first plenum communicates with the openings in use toexhaust substantially all of the flow of cooling gas through theopenings and hence through the array, wherein the divider wall isconfigured such that the first inlet at least partially verticallyoverlaps with the plurality of openings through the array to allow thefirst inlet to admit the gas through the first plenum in a substantiallyhorizontal direction; wherein the plurality of vertically arrangedimpellers are disposed in the heat exchanger chamber verticallyoverlapping with the first inlet, the plurality of vertically arrangedimpellers configured to horizontally impel a substantially uniformcurtain of gas through the heat exchanger and to the first inlet andimpel the gas substantially horizontally during an entire circulation ofthe gas through the equipment chamber and the heat exchanger chamber,and the flow of gas through the array is substantially parallel andopposed to the flow of gas through the heat exchanger, wherein theimpeller array is configured such that any of the impellers can bereplaced while the other impellers continue to operate.
 28. The systemof claim 27 wherein each of the impellers is removably connected to thecabinet with quick-release fittings.